The Story That Explains Threads
A naive waiter would stand next to the kitchen door for 10 minutes doing nothing, then serve Table 1, then walk to Table 2 for the next order. Three tables would take 30 minutes.
A smart waiter doesn't wait. He drops the order at the kitchen and immediately walks to Table 2 to take another order, then Table 3, then refills water at Table 4 — using the 10 minutes of cooking time to serve everyone else. When Table 1's pasta is ready, he delivers it, then keeps juggling. Same one waiter, but three tables served in ~12 minutes instead of 30.
That is exactly what Python threading is: one worker (a single Python interpreter) that cleverly switches between many tasks while any of them is waiting on something slow — a network response, a database query, a file read, a subprocess. It's not five cooks working in parallel; it's one smart waiter turning wasted waiting time into productive work.
A process is a running program — an isolated address space with its own memory. A thread is a single sequence of execution inside a process. A process always has at least one thread (the main thread). Adding more threads lets a single process juggle many tasks by time-sharing — the Python interpreter rapidly switches between them, giving each a small slice of CPU time. From the outside they look concurrent; internally, at any single instant, only one is running.
A thread that is waiting on I/O — a network reply, a disk read, a
database query, time.sleep(), input() — is doing no CPU work.
Python takes that idle moment and lets another thread run instead. Threads don't make
your CPU compute faster; they stop your program from sitting idle while it waits.
That's why threading gives huge speedups on I/O-bound programs and almost none on
pure-math CPU-bound programs.
Single-Threaded vs Multi-Threaded — The Visual Difference
Sequential execution wastes time sitting idle during each I/O wait. Threading overlaps those waits — one worker keeps switching to whichever task is ready to make progress, so the wall-clock total shrinks to the slowest single task instead of the sum.
_thread Module vs threading Module
Python ships with two threading APIs. Understanding why matters — you will encounter both in old code and interview questions.
| Feature | Status |
|---|---|
| Origin | Python 1.x — very old |
| API surface | Bare bones (2 functions) |
| Named threads | No |
| join() / wait | Must build manually |
| Locks & Events | Only basic Lock |
| Should you use it? | NO — only for legacy code |
| Feature | Status |
|---|---|
| Origin | Python 2.4+ — built on _thread |
| API surface | Rich: Thread, Lock, Event, Semaphore, Timer, Barrier |
| Named threads | Yes — Thread(name="worker-1") |
| join() / wait | Built in with timeout |
| Locks & Events | Full synchronization toolkit |
| Should you use it? | YES — always |
# --- The old _thread way (avoid) ---
import _thread, time
def worker(name):
print(f"{name} started")
time.sleep(1)
print(f"{name} done")
_thread.start_new_thread(worker, ("t1",))
time.sleep(2) # have to sleep — no join() available
# --- The modern threading way (use this) ---
import threading, time
def worker(name):
print(f"{name} started")
time.sleep(1)
print(f"{name} done")
t = threading.Thread(target=worker, args=("t1",), name="worker-1")
t.start()
t.join() # proper synchronization
In Python 3 the module was renamed from thread to
_thread (underscore prefix) deliberately — it signals "internal, don't use".
Reach for threading every single time.
Creating Your First Thread — The Function-Based Way
The simplest way to spawn a thread: pass a target function to threading.Thread,
then call .start(). Python creates a new OS thread, that thread calls your
function, and control returns to your main thread immediately.
import threading, time
def count_down(name, n):
for i in range(n, 0, -1):
print(f"{name}: {i}")
time.sleep(1)
print(f"{name}: BLAST OFF!")
# Create two threads that run concurrently
t1 = threading.Thread(target=count_down, args=("Rocket-A", 3))
t2 = threading.Thread(target=count_down, args=("Rocket-B", 3))
t1.start() # begins concurrently
t2.start() # begins concurrently
t1.join() # wait for t1 to finish
t2.join() # wait for t2 to finish
print("Both rockets launched.")
Both countdowns print in an interleaved pattern — that's proof they're truly running concurrently. If order matters, use synchronization primitives (locks, events). Threads are inherently non-deterministic in their output order.
The Thread Class — Constructor & Attributes
| Parameter | Purpose | Typical Use |
|---|---|---|
target | Callable the thread will execute | The function you want to run |
args | Tuple of positional arguments | args=(url, timeout) |
kwargs | Dict of keyword arguments | kwargs={'verbose': True} |
name | Human-readable label | Great for debugging & logs |
group | Reserved for future use | Always None |
| Method / Property | What It Does |
|---|---|
t.start() | Launches the thread — runs target in the NEW thread |
t.run() | Runs target in the CURRENT thread (do NOT call directly) |
t.join(timeout=None) | Blocks caller until this thread finishes (or timeout) |
t.is_alive() | True if thread has started but not yet terminated |
t.name | The thread's label (mutable) |
t.ident | OS thread identifier (int, None until started) |
t.native_id | Kernel-level thread id (Python 3.8+) |
import threading, time
def work(seconds):
time.sleep(seconds)
t = threading.Thread(
target=work,
args=(2,),
name="io-worker-01"
)
print(f"name: {t.name}")
print(f"alive? {t.is_alive()}") # False — not started
print(f"ident: {t.ident}") # None — not started
t.start()
print(f"alive? {t.is_alive()}") # True
print(f"ident: {t.ident}") # now an integer
print(f"native_id: {t.native_id}") # kernel TID
t.join()
print(f"alive? {t.is_alive()}") # False — finished
start() vs run() — The Critical Distinction
This is the single most misunderstood thing about Python threading. Both methods exist, both call your target function — but only one of them actually creates a new thread. Confusing them silently kills your concurrency.
start() asks the OS to spawn a new thread and calls run() inside it. Calling run() yourself skips the OS spawn — it's just a normal method call in your current thread.
import threading, time
def slow_task():
print(f" [{threading.current_thread().name}] working…")
time.sleep(2)
print(f" [{threading.current_thread().name}] done")
# === CORRECT: use start() ===
print("\n--- with start() ---")
start_time = time.time()
threads = [threading.Thread(target=slow_task, name=f"t{i}") for i in range(3)]
for t in threads: t.start()
for t in threads: t.join()
print(f"took {time.time() - start_time:.1f}s")
# === WRONG: use run() ===
print("\n--- with run() ---")
start_time = time.time()
threads = [threading.Thread(target=slow_task, name=f"t{i}") for i in range(3)]
for t in threads: t.run() # NO new threads created!
print(f"took {time.time() - start_time:.1f}s")
Always call t.start(). Never call t.run()
yourself. run() is designed to be called by Python internally
inside the freshly spawned thread. If you call it manually, everything runs in the
current thread — you get zero concurrency and the same result as calling the function
directly.
Passing Arguments — args and kwargs
Pass positional args as a tuple to args= and keyword args as a dict to
kwargs=. Remember the trailing comma when passing a single positional arg —
args=("only",), not args=("only").
import threading, time
def download(url, timeout, retries=3, verbose=False):
if verbose:
print(f"GET {url} (timeout={timeout}s, retries={retries})")
time.sleep(0.5)
return f"done: {url}"
# Positional args as a tuple, keyword args as a dict
t = threading.Thread(
target=download,
args=("https://api.example.com", 10), # url, timeout
kwargs={"retries": 5, "verbose": True}
)
t.start()
t.join()
# Common bug — forgetting the trailing comma
# args=("hello") is a str, not a tuple → TypeError
# args=("hello",) is a 1-tuple → works
Thread ignores whatever your target function returns. To collect results,
write to a shared container (protected with a Lock, or a thread-safe Queue),
or use concurrent.futures.ThreadPoolExecutor which returns
Future objects.
Subclassing Thread — The Class-Based Way
When your thread needs internal state, helper methods, or a clear identity, subclass
Thread and override run(). This is the object-oriented
alternative to passing a function.
threading.Thread. Now every instance is a full-featured
thread object.
run() runs
in the new thread when start() is called.
run() yourself. start() spawns the new OS thread
which then calls run() internally.
import threading, time
class Downloader(threading.Thread):
def __init__(self, url, retries=3):
super().__init__(name=f"dl-{url[-6:]}") # call parent init!
self.url = url
self.retries = retries
self.result = None # place to store output
self.error = None
def run(self): # override run()
try:
print(f"[{self.name}] fetching…")
time.sleep(1) # pretend it's a real request
self.result = f"<html of {self.url}>"
except Exception as e:
self.error = e
# Use it exactly like a Thread
urls = ["example.com/a", "example.com/b", "example.com/c"]
workers = [Downloader(u) for u in urls]
for w in workers: w.start() # start(), never run()
for w in workers: w.join()
for w in workers:
if w.error:
print(f"{w.name} FAILED: {w.error}")
else:
print(f"{w.name} → {w.result}")
If you override __init__, you MUST call super().__init__()
first. Otherwise Thread's internal machinery (name, ident, alive flag) is
never set up, and start() raises
RuntimeError: thread.__init__() not called.
Function vs Class — When to Use Which
| Aspect | Function-Based | Class-Based (subclass) |
|---|---|---|
| Boilerplate | Very low — one line | A whole class |
| Storing state | Awkward — globals or closures | Natural — self.attr |
| Returning results | Via shared container/Queue | Store on self.result after run() |
| Reuse (helper methods) | Not built-in | Add methods like any class |
| One-off scripts | Perfect fit | Overkill |
| Worker services / patterns | Feels ad-hoc | Clean, testable |
Start with the function-based approach for quick scripts. Move to subclassing Thread when you need persistent state, helper methods, or reusable worker types.
Thread Lifecycle — The Four States
A thread flows one-way: New → Runnable → (Running ↔ Runnable)* → Terminated. Once terminated, you cannot call start() again on the same object.
The join() Method — Waiting for Threads to Finish
join() is the single most important method for controlling threaded programs.
It's how the main thread (or any thread) says: "pause me right here until that other
thread has finished its work". Without join(), threads run wildly in
parallel with no coordination — and your main thread might exit before the workers even
print their first line.
What join() Actually Does
join() puts the calling thread into a wait state — using zero CPU — until the target
thread reaches its TERMINATED state. Then the OS wakes the caller up and it continues.
Why join() Exists — What Happens Without It
import threading, time
def worker():
print("worker: starting")
time.sleep(2)
print("worker: finished")
print("main: launching thread")
t = threading.Thread(target=worker)
t.start()
# NO join() — main thread races ahead
print("main: done")
Notice the problem — main: done printed while the worker was still running.
In a bigger script this means the main thread might try to use results the worker hasn't
produced yet, close files it hasn't finished writing, or shut down the program before
work completes.
import threading, time
def worker():
print("worker: starting")
time.sleep(2)
print("worker: finished")
print("main: launching thread")
t = threading.Thread(target=worker)
t.start()
t.join() # wait here until worker is done
print("main: done")
Think of join() as a checkpoint that says "I refuse to continue past this
line until thread t has finished". The caller (usually main) stops. The
worker keeps going. When the worker exits, the checkpoint releases and main resumes.
join(timeout=N) — Wait Only So Long
Sometimes you don't want to wait forever. Pass a timeout in seconds and
join() returns after that many seconds regardless of whether the thread
finished. You then check t.is_alive() to know what happened.
import threading, time
def very_slow_task():
time.sleep(10)
print("task finished after 10s")
t = threading.Thread(target=very_slow_task)
t.start()
print("main: waiting up to 3 seconds…")
t.join(timeout=3) # give up after 3s
if t.is_alive():
print("main: worker still running — moving on without it")
else:
print("main: worker finished on time")
print("main: continuing with other work")
A common misconception: join(timeout=3) does not stop the thread after
3 seconds — it only stops waiting. The thread keeps running in the background.
Python has no built-in way to forcibly kill a thread. You need cooperative shutdown
via threading.Event or a shared flag.
Joining Multiple Threads — The Right Pattern
When you have several threads, always start all of them first, then join them all. Doing start-join-start-join makes them run one after another and destroys your concurrency.
for t in threads: |
t.start() |
t.join() ← blocks here! |
| Each thread starts, main waits for it, |
| only then next thread starts. |
for t in threads: t.start() |
for t in threads: t.join() |
| All threads start together, all run |
| concurrently, main joins each in order. |
| Total = slowest single thread. |
import threading, time
def task(n):
time.sleep(2)
print(f"task {n} done")
threads = [threading.Thread(target=task, args=(i,)) for i in range(5)]
# === WRONG — start & join in same loop (10 seconds total) ===
start = time.time()
for t in threads:
t.start()
t.join() # blocks before next thread starts
print(f"wrong way: {time.time() - start:.1f}s")
# === RIGHT — separate loops (2 seconds total) ===
threads = [threading.Thread(target=task, args=(i,)) for i in range(5)]
start = time.time()
for t in threads: t.start() # fire them all off first
for t in threads: t.join() # then wait for all
print(f"right way: {time.time() - start:.1f}s")
Collecting Results After join()
Since Thread targets can't return values directly, the common pattern is to
store results on the thread object itself (when subclassing) or in a shared list.
join() guarantees the worker is finished, so it's safe to read the result
after join() returns.
import threading, time
class SumWorker(threading.Thread):
def __init__(self, numbers):
super().__init__()
self.numbers = numbers
self.result = None
def run(self):
time.sleep(1) # simulate work
self.result = sum(self.numbers)
# Split the work across 3 workers
w1 = SumWorker([1, 2, 3, 4, 5])
w2 = SumWorker([6, 7, 8, 9, 10])
w3 = SumWorker([11, 12, 13, 14, 15])
for w in (w1, w2, w3): w.start()
for w in (w1, w2, w3): w.join() # AFTER this line, results are ready
total = w1.result + w2.result + w3.result
print(f"partials: {w1.result}, {w2.result}, {w3.result}")
print(f"total: {total}")
After t.join() returns, everything that thread t wrote to
shared memory (like self.result) is guaranteed visible to the joining
thread. You don't need a lock to read results after a successful join.
join() Cannot Restart a Thread
Once a thread has finished, calling start() on it again raises
RuntimeError. Thread objects are single-use — create a fresh one
for each new task.
import threading, time
t = threading.Thread(target=lambda: time.sleep(1))
t.start()
t.join() # wait for it to finish
print(f"alive after join? {t.is_alive()}") # False
try:
t.start() # try to reuse
except RuntimeError as e:
print(f"error: {e}")
# You can, however, call join() as many times as you like
t.join() # safe — returns immediately
t.join() # also safe
print("multiple joins on a finished thread: OK")
Never join() a Thread From Itself
A thread joining itself is a deadlock — it's asking to wait for itself to finish, which
can only happen after it has finished waiting for itself, which... you see the problem.
Python catches this and raises RuntimeError.
import threading
def bad_worker():
# current_thread() returns the Thread object we're inside
me = threading.current_thread()
try:
me.join() # waiting for myself — nonsense!
except RuntimeError as e:
print(f"caught: {e}")
threading.Thread(target=bad_worker).start()
join() Return Value
join() always returns None — it does not return the target
function's result. If you want the result, store it on the thread object or in a shared
container. This surprises many beginners.
import threading
def compute():
return 42 # this return value is DROPPED
t = threading.Thread(target=compute)
t.start()
result = t.join() # join returns None, not 42
print(f"join returned: {result}") # None
print("to capture the return value, use a subclass or a shared container")
Practical Example — Parallel File Processing with Progress
import threading, time, random
class FileProcessor(threading.Thread):
def __init__(self, filename):
super().__init__(name=f"proc-{filename}")
self.filename = filename
self.bytes_read = 0
self.done = False
def run(self):
# pretend we're processing a file of random size
size = random.randint(1, 4)
time.sleep(size)
self.bytes_read = size * 1024 * 1024
self.done = True
# Kick off 5 workers
files = ["log01.txt", "log02.txt", "log03.txt", "log04.txt", "log05.txt"]
workers = [FileProcessor(f) for f in files]
for w in workers: w.start()
# Poll with short join timeouts to show live progress
while any(w.is_alive() for w in workers):
for w in workers:
w.join(timeout=0.5) # tiny wait per worker
still_running = sum(1 for w in workers if w.is_alive())
print(f"[progress] {len(workers) - still_running}/{len(workers)} finished")
# All done — safe to read results
print("\n=== Summary ===")
for w in workers:
print(f"{w.filename}: {w.bytes_read / 1024 / 1024:.1f} MB")
Calling join(timeout=0.5) in a loop is the idiomatic way to build progress
bars, status prints, or heartbeat checks while background work runs. It's more efficient
than polling is_alive() in a tight loop because the OS actually pauses
your thread instead of spinning.
join() Summary Cheat Sheet
| Call | Behaviour | Returns |
|---|---|---|
t.join() | Wait forever until t finishes | None |
t.join(timeout=5) | Wait at most 5 seconds | None (check is_alive()) |
t.join(0) | Return immediately — same as a status check | None |
t.join() before t.start() | Raises RuntimeError | — |
t.join() after t finished | Returns immediately (safe to call again) | None |
t.join() from inside t itself | Raises RuntimeError — deadlock guard | — |
join() non-daemon threads before letting the
program exit. Without it, main can race ahead and try to use results that don't
exist yet.
start() them all, then another to join() them all. Never
interleave start/join in the same loop or you serialize your work.
join(timeout=N) does not kill the thread. It only stops
waiting. Check t.is_alive() afterwards to know whether the thread
finished or the timeout hit.
join() always returns None. To capture a
worker's result, subclass Thread and store it on self.result,
or use a shared list / queue.Queue. Read the result AFTER
join() returns.
t.join() from within thread t itself. Python
raises RuntimeError — but if it didn't, it would deadlock forever.
join() multiple times on the same finished thread is
safe — it returns immediately. Calling start() a second
time is not — RuntimeError: threads can only be started once.