Python Daemon Threads, Synchronization, Lock & RLock — Safe Concurrent Programming

The Story That Explains Synchronization

This tutorial covers the two most-used tools for safe threading: daemon threads (background helpers that die when the program exits) and the two lock types — Lock and RLock — that stop threads from corrupting shared data.

Daemon Threads — Background Helpers That Die With Main

Every thread you create is either a non-daemon thread (the default) or a daemon thread. The difference is only visible at program shutdown:

Creating a Daemon Thread

import threading, time

def background_monitor():
    # Infinite loop — would never let a non-daemon program exit
    while True:
        time.sleep(1)
        print("[monitor] heartbeat")

# Way 1: set in constructor (recommended)
mon = threading.Thread(target=background_monitor, daemon=True)

# Way 2: set as attribute BEFORE start()
# mon = threading.Thread(target=background_monitor)
# mon.daemon = True

mon.start()

time.sleep(3.5)
print("main exiting — daemon monitor dies automatically")

Classic Textbook Example — Daemon Killed Before Finishing

Here's the shortest, clearest demo of daemon behaviour. We spawn two threads — one non-daemon (sleeps 3 seconds) and one daemon (sleeps 4 seconds). Watch what happens: the daemon never gets to print its exit message because the program terminates the moment the non-daemon finishes.

from threading import Thread
import time

def non_d():
    print("Non daemon enters")
    time.sleep(3)                    # will finish at t = 3s
    print("nonDaemon exits")

def d():
    print("daemon enters")
    time.sleep(4)                    # would finish at t = 4s… but gets killed at 3s
    print("Daemon exits")             # NEVER prints

nd = Thread(target=non_d)
nd.start()

d1 = Thread(target=d)
d1.setDaemon(True)                # old-style setter; equivalent to d1.daemon = True
d1.start()

Try This Variation — Reverse the Times

Swap the sleep durations: make the non-daemon sleep 4 seconds and the daemon sleep 3. Now the daemon does get to print its exit message, because the non-daemon is still keeping the program alive when the daemon naturally finishes.

from threading import Thread
import time

def non_d():
    print("Non daemon enters")
    time.sleep(4)                    # now the LONGER one
    print("nonDaemon exits")

def d():
    print("daemon enters")
    time.sleep(3)                    # now the SHORTER one
    print("Daemon exits")             # now it DOES print

nd = Thread(target=non_d); nd.start()
d1 = Thread(target=d);     d1.daemon = True; d1.start()

The Danger of Daemon Threads — No Clean Shutdown

import threading, time

def important_writer():
    try:
        with open("data.txt", "w") as f:
            for i in range(100):
                f.write(f"line {i}\n")
                time.sleep(0.05)          # pretend slow disk
        print("wrote all 100 lines")
    except Exception as e:
        print(f"interrupted: {e}")

# If we mark this as daemon, main exits mid-write → file is truncated!
t = threading.Thread(target=important_writer, daemon=True)
t.start()

time.sleep(1)          # only ~20 lines written by now
print("main done — daemon writer gets KILLED mid-write")
# Result: data.txt contains partial data, no "wrote all 100 lines" message

When to Use Which

The Race Condition Problem

Before we introduce locks, let's see the exact problem they solve. A race condition happens when two or more threads read-modify-write the same value, and the interleaving of their operations produces a wrong result.

Race Condition Demo — Broken Code

import threading

counter = 0

def increment():
    global counter
    for _ in range(100_000):
        counter += 1              # NOT atomic!

threads = [threading.Thread(target=increment) for _ in range(10)]

for t in threads: t.start()
for t in threads: t.join()

print(f"expected: {10 * 100_000}")
print(f"got:      {counter}")
print(f"lost:     {10 * 100_000 - counter} increments")

Lock — The Turn-Taking Token

threading.Lock is Python's most basic synchronization primitive. It has just two states — unlocked and locked — and two operations — acquire and release. The rule is simple: only one thread can hold the lock at a time. Everyone else who tries to acquire it is put to sleep until the current holder releases.

Fixing the Race Condition

import threading

counter = 0
lock    = threading.Lock()

def increment():
    global counter
    for _ in range(100_000):
        lock.acquire()                # wait if someone else holds it
        counter += 1                # now safe — we're alone
        lock.release()                # let the next thread in

threads = [threading.Thread(target=increment) for _ in range(10)]

for t in threads: t.start()
for t in threads: t.join()

print(f"expected: {10 * 100_000}")
print(f"got:      {counter}")                # exactly 1_000_000

The Right Way — with Statement

Calling acquire() and release() manually is fragile — if your code raises an exception between them, the lock stays locked forever and every waiting thread deadlocks. Use the with statement instead. It calls acquire() on entry and guarantees release() on exit, even on exceptions.

import threading

counter = 0
lock    = threading.Lock()

def increment():
    global counter
    for _ in range(100_000):
        with lock:                     # acquires on enter, releases on exit
            counter += 1                # exception-safe

threads = [threading.Thread(target=increment) for _ in range(10)]
for t in threads: t.start()
for t in threads: t.join()

print(f"counter = {counter}")

Lock Methods Reference

import threading, time

lock = threading.Lock()

def try_get_lock(worker_id):
    if lock.acquire(blocking=False):
        try:
            print(f"worker {worker_id} got the lock, working…")
            time.sleep(1)
        finally:
            lock.release()
    else:
        print(f"worker {worker_id} could not get lock — skipping")

threads = [threading.Thread(target=try_get_lock, args=(i,)) for i in range(4)]
for t in threads: t.start()
for t in threads: t.join()

Deadlock — When Locks Attack

A deadlock is when two or more threads are stuck waiting for locks the others hold. Nothing crashes, nothing prints — the program just freezes forever.

import threading, time

lock1 = threading.Lock()
lock2 = threading.Lock()

def thread_a():
    with lock1:
        print("A got lock1")
        time.sleep(0.1)                # give B time to grab lock2
        with lock2:                     # will wait FOREVER
            print("A got lock2 (never prints)")

def thread_b():
    with lock2:
        print("B got lock2")
        time.sleep(0.1)
        with lock1:                     # will wait FOREVER
            print("B got lock1 (never prints)")

threading.Thread(target=thread_a).start()
threading.Thread(target=thread_b).start()
# Program hangs — Ctrl-C to exit

RLock — The Reentrant Lock

Sometimes a thread needs to acquire the same lock more than once — usually because one locked function calls another locked function. With a plain Lock, this deadlocks the thread against itself. RLock (reentrant lock) solves it by remembering which thread owns it and how many times it's been acquired.

The Problem Plain Lock Creates

import threading

lock = threading.Lock()

def deposit(amount):
    with lock:                          # acquire #1
        print(f"deposit {amount}")
        audit(f"deposit of {amount}")   # calls audit — which also locks!

def audit(msg):
    with lock:                          # acquire #2 — SELF-DEADLOCK
        print(f"[audit] {msg}")

t = threading.Thread(target=deposit, args=(100,))
t.start()
t.join(timeout=2)
print(f"alive after 2s? {t.is_alive()}")     # True — deadlocked

The RLock Fix

import threading

rlock = threading.RLock()               # reentrant version

def deposit(amount):
    with rlock:                         # count = 1
        print(f"deposit {amount}")
        audit(f"deposit of {amount}")

def audit(msg):
    with rlock:                         # count = 2 — same thread, OK
        print(f"[audit] {msg}")

t = threading.Thread(target=deposit, args=(100,))
t.start()
t.join()
print("finished cleanly")

Problem #1 — Nested Function Calls (Where RLock Shines)

Here's a textbook demonstration of the exact scenario RLock was designed for. A single thread runs the function all(), which acquires the lock and then calls first() and second() — both of which also acquire the same lock. With a plain Lock this self-deadlocks immediately. With RLock, the same thread is allowed to re-enter.

import threading
lock = threading.RLock()

def first(n):
    print("In first", n)
    lock.acquire()                # count: 1 → 2  (same thread, allowed)
    a = 12 + n
    lock.release()                # count: 2 → 1
    print(a)

def second(n):
    lock.acquire()                # count: 1 → 2  (same thread, allowed)
    b = 12 + n
    lock.release()                # count: 2 → 1
    print(b)

def all():
    print("start")
    lock.acquire()                # count: 0 → 1  (thread owns it now)
    first(2)                    # re-acquires + releases inside
    second(3)                   # re-acquires + releases inside
    lock.release()                # count: 1 → 0  (truly freed)

th1 = threading.Thread(target=all)
th1.start()

Step-by-Step — The Recursion Counter in Action

Problem #2 — Cross-Thread Release (Where RLock Refuses)

Now here's a case that looks like RLock should save us — but it can't, because RLock deliberately enforces ownership. One thread acquires the lock, and a different thread tries to release it. Watch what happens.

import threading
import time
import datetime

lock = threading.RLock()
t1   = datetime.datetime.now()

def second(n):
    lock.acquire()                # acquires but NEVER releases!
    print(n)

def third():
    time.sleep(5)
    lock.release()                # releases a lock this thread never acquired!
    print("Thread3")

th1 = threading.Thread(target=second, args=("Thread1",))
th1.start()

th2 = threading.Thread(target=second, args=("Thread2",))
th2.start()

th3 = threading.Thread(target=third)
th3.start()

th1.join()
th2.join()
th3.join()
t2 = datetime.datetime.now()
print("Total time", t2 - t1)

What Actually Happens — Trace by Trace

The Fix — Each Function Manages Its Own Lock Lifecycle

import threading, time, datetime

lock = threading.RLock()
t1   = datetime.datetime.now()

def second(n):
    with lock:                    # acquire + guaranteed release
        print(n)
        time.sleep(1)              # pretend we're doing critical work

def third():
    time.sleep(2)
    with lock:                    # properly acquire before touching shared state
        print("Thread3")

threads = [
    threading.Thread(target=second, args=("Thread1",)),
    threading.Thread(target=second, args=("Thread2",)),
    threading.Thread(target=third),
]
for t in threads: t.start()
for t in threads: t.join()

print("Total time", datetime.datetime.now() - t1)

Lock vs RLock — Side by Side

Rule of Thumb

Real-World Example — Thread-Safe Bank Account

Let's build a BankAccount class where multiple threads can deposit, withdraw, and transfer between accounts without corrupting the balance. This shows why RLock is the natural choice: transfer() calls withdraw() and deposit() — nested locking.

import threading, time

class BankAccount:
    def __init__(self, name, balance=0):
        self.name    = name
        self.balance = balance
        self.lock    = threading.RLock()    # reentrant — allows nesting

    def deposit(self, amount):
        with self.lock:
            new_bal = self.balance + amount
            time.sleep(0.001)              # simulate DB latency
            self.balance = new_bal
            print(f"  [+] {self.name} +{amount} → {self.balance}")

    def withdraw(self, amount):
        with self.lock:
            if amount > self.balance:
                raise ValueError(f"{self.name}: insufficient funds")
            new_bal = self.balance - amount
            time.sleep(0.001)
            self.balance = new_bal
            print(f"  [-] {self.name} -{amount} → {self.balance}")

    def transfer(self, to_account, amount):
        # RLock lets us acquire our own lock twice — deposit/withdraw re-acquire it
        with self.lock:                       # outer lock on source
            with to_account.lock:             # lock on destination
                self.withdraw(amount)          # re-acquires self.lock — OK with RLock
                to_account.deposit(amount)     # re-acquires to_account.lock — OK
                print(f"  => transferred {amount} {self.name} → {to_account.name}")

# Two accounts, many concurrent operations
alice = BankAccount("alice", 1000)
bob   = BankAccount("bob",   1000)

def busy_customer():
    for _ in range(5):
        alice.deposit(10)
        bob.withdraw(5)
        alice.transfer(bob, 20)

threads = [threading.Thread(target=busy_customer, name=f"cust-{i}")
           for i in range(3)]
for t in threads: t.start()
for t in threads: t.join()

print(f"\nfinal:  alice={alice.balance}  bob={bob.balance}")
print(f"total: {alice.balance + bob.balance}   (should be 2000 — money is conserved)")

Common Pitfalls

Golden Rules