Python GIL & Threading with Queue

The Story That Explains The GIL

The GIL is a mutex (mutual exclusion lock) inside CPython that protects access to Python objects. It prevents multiple native threads from executing Python bytecode simultaneously. This design keeps CPython simple and fast for single-threaded workloads, but it fundamentally limits CPU-bound multithreaded performance.

What Exactly Is The GIL?

The Global Interpreter Lock is a single, process-wide mutex in the CPython interpreter. Every Python thread must acquire the GIL before it can execute a single bytecode instruction. When one thread holds it, all others are blocked at the Python level.

Why Does The GIL Exist?

The GIL was introduced in the early 1990s to solve one problem: memory management safety. CPython uses reference counting for garbage collection. Without a global lock, two threads could increment or decrement a refcount at the same time, corrupting memory and crashing the interpreter.

Behind the Scenes — Threads & PyThreadState

When you call threading.Thread(target=…).start(), Python does surprisingly little. It creates a small C data structure called PyThreadState, launches a native OS thread (a POSIX pthread on Linux/macOS, a Windows thread on Windows), and hands it off. Thread scheduling itself is done entirely by the operating system — Python has no thread scheduler of its own.

The PyThreadState Structure

Each Python thread carries its own state — a stack frame, recursion depth, exception info, tick counter, and tracing hooks. A global variable _PyThreadState_Current always points to the state of whichever thread is currently running.

/* Python/pystate.h — simplified */
typedef struct _ts {
    struct _ts       *next;
    PyInterpreterState *interp;
    struct _frame   *frame;          // current stack frame
    int                 recursion_depth;
    int                 tracing;
    Py_tracefunc        c_profilefunc;
    Py_tracefunc        c_tracefunc;
    PyObject           *curexc_type;
    PyObject           *curexc_value;
    PyObject           *curexc_traceback;
    PyObject           *dict;            // thread-local storage
    int                 tick_counter;    // used by the check interval
    int                 gilstate_counter;
    PyObject           *async_exc;
    long                thread_id;
} PyThreadState;

/* Points to currently executing thread's state */
PyThreadState *_PyThreadState_Current = NULL;

Ticks, Check Intervals & The Modern Switch

How does Python decide when to release the GIL from a CPU-bound thread that never does I/O? It periodically performs a "check". The scheme changed dramatically between Python 2 and Python 3.2.

What Is a "Tick"?

A tick loosely maps to one Python virtual-machine instruction. You can see them by disassembling any function with dis:

>>> import dis
>>> def countdown(n):
...     while n > 0:
...         print(n)
...         n -= 1
>>> dis.dis(countdown)
  0 SETUP_LOOP           33 (to 36)
  3 LOAD_FAST             0 (n)             # <- Tick 1
  6 LOAD_CONST            1 (0)
  9 COMPARE_OP            4 (>)
 12 JUMP_IF_FALSE        19 (to 34)
 15 POP_TOP
 16 LOAD_FAST             0 (n)
 19 PRINT_ITEM
 20 PRINT_NEWLINE                          # <- Tick 2
 21 LOAD_FAST             0 (n)
 24 LOAD_CONST            2 (1)                # <- Tick 3
 27 INPLACE_SUBTRACT
 28 STORE_FAST            0 (n)                # <- Tick 4
 31 JUMP_ABSOLUTE         3

The Check Itself — Simplified From ceval.c

/* Python/ceval.c — the heart of the interpreter loop */
if (--_Py_Ticker < 0) {
    _Py_Ticker = _Py_CheckInterval;

    if (things_to_do) {
        if (Py_MakePendingCalls() < 0) { ... }
    }

    if (interpreter_lock) {
        /* Give another thread a chance */
        PyThread_release_lock(interpreter_lock);

        /* Other threads may run now */

        PyThread_acquire_lock(interpreter_lock, 1);
    }
}

GIL Behavior — Cooperative I/O Release

For I/O-bound threads, the GIL is beautifully simple: a thread holds the GIL while running Python code, but releases it whenever it blocks for I/O. Any other ready thread immediately grabs the lock and runs.

CPU-Bound vs I/O-Bound — When The GIL Bites

The Beazley Experiment — When 2 Threads Are SLOWER Than 1

Recreating The Experiment

import time
from threading import Thread

def count(n):
    while n > 0:
        n -= 1

COUNT = 100_000_000

# --- Sequential ---
start = time.perf_counter()
count(COUNT)
count(COUNT)
print(f"Sequential : {time.perf_counter() - start:.1f}s")

# --- Two threads (each does half? no — each does full!) ---
start = time.perf_counter()
t1 = Thread(target=count, args=(COUNT,))
t2 = Thread(target=count, args=(COUNT,))
t1.start(); t2.start()
t1.join();  t2.join()
print(f"Threaded   : {time.perf_counter() - start:.1f}s")

The Multicore GIL Battle

On a single core, the OS runs one thread at a time — the GIL is barely contested. On multicore, the OS gleefully schedules multiple threads on different cores at the same time, and they immediately start fighting over the one interpreter lock.

Beazley's Instrumented Trace — The GIL Battle Live

# t1, t2 = thread ids. Middle number = ticks remaining. Right = total checks.
t2 100 5392 ENTRY
t2 100 5392 ACQUIRE
t2 100 5393 RELEASE
t1 100 5393 ACQUIRE     # thread switch — t1 wins
t2 100 5393 ENTRY
t2  27 5393 BUSY        # t2 tries to run but GIL taken by t1
t1 100 5394 RELEASE
t1 100 5394 ENTRY       # t1 releases and IMMEDIATELY re-enters
t1 100 5394 ACQUIRE     # t1 reacquires before t2 can wake
t2  74 5394 RETRY       # t2 wakes up, fails, goes back to sleep
t1 100 5395 RELEASE
t1 100 5395 ENTRY
t1 100 5395 ACQUIRE     # t1 wins AGAIN
t2  83 5395 RETRY       # t2 fails AGAIN
...  # hundreds of failed retries before t2 gets a turn

Signaling Overhead — Where All The Time Goes

The GIL is not a simple mutex. On Unix it's implemented as either a POSIX unnamed semaphore or a pthreads condition variable. Every acquire/release involves signaling — and signaling costs system calls.

Priority Inversion — When CPU Threads Starve I/O Threads

Here's the truly ugly one. Imagine a mixed workload: one CPU-bound worker crunching numbers, one I/O-bound worker sitting on a socket waiting for a packet. The packet arrives — the I/O thread wants to run right now — but it can't get the GIL.

Signals, Ctrl-C & Frozen Threaded Programs

Safe Pattern — Interruptible Threaded Program

import threading
import time

stop_event = threading.Event()

def worker():
    while not stop_event.is_set():
        # do real work, but check stop_event periodically
        time.sleep(0.1)
    print("Worker exiting cleanly.")

t = threading.Thread(target=worker)
t.start()

try:
    # Main thread stays responsive to signals
    while t.is_alive():
        t.join(timeout=0.5)     # timeout lets main thread wake to check signals
except KeyboardInterrupt:
    print("\nCtrl-C received — asking worker to stop")
    stop_event.set()
    t.join()

Proving The GIL — A CPU-Bound Benchmark

import time
import threading
from multiprocessing import Process

# A pure Python CPU-bound task
def count_down(n):
    while n > 0:
        n -= 1

COUNT = 50_000_000

# --- 1. Single thread (baseline) ---
start = time.perf_counter()
count_down(COUNT)
print(f"Single thread:  {time.perf_counter() - start:.2f}s")

# --- 2. Two threads (splitting the work) ---
start = time.perf_counter()
t1 = threading.Thread(target=count_down, args=(COUNT // 2,))
t2 = threading.Thread(target=count_down, args=(COUNT // 2,))
t1.start(); t2.start()
t1.join(); t2.join()
print(f"Two threads:    {time.perf_counter() - start:.2f}s")

# --- 3. Two processes (bypasses the GIL) ---
start = time.perf_counter()
p1 = Process(target=count_down, args=(COUNT // 2,))
p2 = Process(target=count_down, args=(COUNT // 2,))
p1.start(); p2.start()
p1.join(); p2.join()
print(f"Two processes:  {time.perf_counter() - start:.2f}s")

Threads Shine For I/O — Same Benchmark, Different Workload

import time
import threading
import requests

URLS = ["https://httpbin.org/delay/1"] * 10

def fetch(url):
    requests.get(url)

# --- Sequential ---
start = time.perf_counter()
for url in URLS:
    fetch(url)
print(f"Sequential:  {time.perf_counter() - start:.2f}s")

# --- 10 threads in parallel ---
start = time.perf_counter()
threads = [threading.Thread(target=fetch, args=(u,)) for u in URLS]
for t in threads: t.start()
for t in threads: t.join()
print(f"10 threads:  {time.perf_counter() - start:.2f}s")

During requests.get(), the underlying socket call releases the GIL. Other threads immediately grab it and fire their requests. All 10 network calls happen in parallel, and the total time collapses to roughly the time of one call. This is exactly why threads paired with a Queue are perfect for I/O work.

Enter The Queue — Thread-Safe Communication

Queue API — The Essential Methods

Your First Producer-Consumer With Queue

import threading
import queue
import time
import random

# A thread-safe, bounded FIFO queue holding at most 5 items
q = queue.Queue(maxsize=5)

def producer(name):
    for i in range(10):
        item = f"{name}-item-{i}"
        q.put(item)                          # blocks if queue is full
        print(f"[P {name}] produced {item} (queue size={q.qsize()})")
        time.sleep(random.uniform(0.05, 0.2))

def consumer(name):
    while True:
        item = q.get()                        # blocks if queue is empty
        if item is None:                      # poison pill = shutdown signal
            q.task_done()
            break
        print(f"    [C {name}] consumed {item}")
        time.sleep(random.uniform(0.1, 0.3))
        q.task_done()                          # signal completion

# Start 2 producers and 3 consumers
producers = [threading.Thread(target=producer, args=(f"P{i}",)) for i in range(2)]
consumers = [threading.Thread(target=consumer, args=(f"C{i}",)) for i in range(3)]

for t in producers + consumers:
    t.start()

# Wait for producers to finish
for t in producers: t.join()

# Wait until all produced items are processed
q.join()

# Send poison pills to shut consumers down cleanly
for _ in consumers:
    q.put(None)

for t in consumers: t.join()

print("All work complete.")

Real-World Example — Concurrent Web Scraper

Let's build something practical: a scraper that downloads 50 URLs using 10 worker threads. A queue holds the URLs, workers pull and process them, and results go into a second queue.

import threading
import queue
import requests
import time

URL_LIST = [f"https://httpbin.org/delay/1?id={i}" for i in range(50)]
NUM_WORKERS = 10

task_queue   = queue.Queue()
result_queue = queue.Queue()

def worker(worker_id):
    while True:
        url = task_queue.get()
        if url is None:                     # shutdown
            task_queue.task_done()
            break
        try:
            r = requests.get(url, timeout=10)
            result_queue.put((url, r.status_code, len(r.content)))
            print(f"[W{worker_id}] {url} -> {r.status_code}")
        except Exception as e:
            result_queue.put((url, "ERROR", str(e)))
        finally:
            task_queue.task_done()

# Feed the queue
for url in URL_LIST:
    task_queue.put(url)

# Start workers
start = time.perf_counter()
workers = [threading.Thread(target=worker, args=(i,), daemon=True)
           for i in range(NUM_WORKERS)]
for w in workers: w.start()

# Wait for all URLs to be processed
task_queue.join()

# Shutdown workers
for _ in workers:
    task_queue.put(None)
for w in workers: w.join()

elapsed = time.perf_counter() - start

# Drain results
results = []
while not result_queue.empty():
    results.append(result_queue.get())

print(f"\nFetched {len(results)} URLs in {elapsed:.2f}s using {NUM_WORKERS} threads")
print(f"Sequential would take ~{len(URL_LIST)}s")

PriorityQueue — When Some Tasks Matter More

A support-ticket system where paying customers jump the queue. The priority number goes first in the tuple — lowest number wins.

import threading
import queue
import time
from dataclasses import dataclass, field
from typing import Any

# Wrap payload so equal-priority items don't try to compare data
@dataclass(order=True)
class Ticket:
    priority: int
    data: Any = field(compare=False)

pq = queue.PriorityQueue()

# Producers add tickets with varied priority (1=urgent, 5=low)
pq.put(Ticket(3, "Fix typo on homepage"))
pq.put(Ticket(1, "PROD DOWN — payment gateway failing!"))
pq.put(Ticket(5, "Update the copyright year"))
pq.put(Ticket(2, "Enterprise customer login broken"))
pq.put(Ticket(1, "Data breach alert!"))

def handler():
    while not pq.empty():
        ticket = pq.get()
        print(f"[P{ticket.priority}] {ticket.data}")
        pq.task_done()
        time.sleep(0.2)

threading.Thread(target=handler).start()

The Modern Way — ThreadPoolExecutor

concurrent.futures.ThreadPoolExecutor gives you a queue-backed thread pool with a clean API. It manages the queue, worker threads, and result collection for you. For most new code, this is what you should reach for.

from concurrent.futures import ThreadPoolExecutor, as_completed
import requests
import time

urls = [f"https://httpbin.org/delay/1?id={i}" for i in range(20)]

def fetch(url):
    r = requests.get(url, timeout=10)
    return url, r.status_code, len(r.content)

start = time.perf_counter()

# The executor manages an internal queue + worker threads
with ThreadPoolExecutor(max_workers=10) as pool:
    # submit() returns a Future — put a task on the queue
    futures = {pool.submit(fetch, u): u for u in urls}

    # as_completed yields futures as they finish
    for future in as_completed(futures):
        url, code, size = future.result()
        print(f"{code} {size:6d}B  {url}")

print(f"\nDone in {time.perf_counter() - start:.2f}s")

Pipeline Pattern — Chained Queues

Real systems often have multiple stages. Example: download → parse → save. Chain queues between stages and each stage runs in its own thread pool.

import threading
import queue
import time

download_q = queue.Queue()
parse_q    = queue.Queue()
save_q     = queue.Queue()

def downloader():
    while True:
        url = download_q.get()
        if url is None: break
        time.sleep(0.3)                     # simulated network
        html = f"<html>from {url}</html>"
        parse_q.put((url, html))
        download_q.task_done()

def parser():
    while True:
        item = parse_q.get()
        if item is None: break
        url, html = item
        time.sleep(0.1)                     # simulated parsing
        record = {"url": url, "len": len(html)}
        save_q.put(record)
        parse_q.task_done()

def saver():
    while True:
        rec = save_q.get()
        if rec is None: break
        print(f"SAVED {rec}")
        save_q.task_done()

# Start each stage as a thread pool
threads = []
for _ in range(4): threads.append(threading.Thread(target=downloader))
for _ in range(2): threads.append(threading.Thread(target=parser))
for _ in range(1): threads.append(threading.Thread(target=saver))
for t in threads: t.start()

# Feed URLs
for i in range(20):
    download_q.put(f"https://example.com/page{i}")

# Wait for each stage to fully drain, then send poison pills
download_q.join()
for _ in range(4): download_q.put(None)

parse_q.join()
for _ in range(2): parse_q.put(None)

save_q.join()
save_q.put(None)

for t in threads: t.join()
print("Pipeline complete.")

Bypassing The GIL — Your Options

Common Pitfalls And Fixes

def safe_worker(work_q, error_q):
    while True:
        item = work_q.get()
        try:
            if item is None:                # poison pill
                break
            process(item)                    # your real work
        except Exception as e:
            error_q.put((item, repr(e)))    # never let exceptions vanish
        finally:
            work_q.task_done()               # ALWAYS signal completion

Threads vs Processes vs Asyncio — Decision Table

Golden Rules