Concurrency and Multithreading

Concurrency means a program deals with several things at once. Parallelism means it actually does several things at the same instant. They are not the same, and knowing which one your problem needs decides whether threads will help or just add bugs. This guide covers the distinction, the two failure modes, and how the common languages approach it.

The distinction#

A cook making three dishes is concurrent: they start the rice, and while it boils they chop vegetables. One person, several jobs in flight.

Three cooks each making one dish are working in parallel: three jobs genuinely happening at the same moment.

Concurrency is a way of structuring a program. Parallelism is a property of how it executes. A concurrent program can run on one processor core; a parallel one needs several.

Which kind of work is it?#

This single question decides your approach:

Downloading a hundred pages is I/O-bound: the program spends almost all its time waiting for replies. Doing them concurrently is a huge win, and one core is plenty. Resizing a hundred images is CPU-bound: the only way to go faster is more cores.

I/O-bound work#

import asyncio, aiohttp

async def fetch(session, url):
    async with session.get(url) as response:
        return len(await response.text())


async def main(urls):
    async with aiohttp.ClientSession() as session:
        results = await asyncio.gather(*(fetch(session, u) for u in urls))
    return results


# Twenty requests take about as long as the slowest one,
# not the sum of all twenty.

The same idea in JavaScript, which is concurrent by default:

const results = await Promise.all(urls.map(u => fetch(u).then(r => r.text())));

Nothing here runs in parallel. There is one thread, and it switches to other work whenever something is waiting. That is enough, because waiting is what the program was doing anyway.

CPU-bound work#

from concurrent.futures import ProcessPoolExecutor


def heavy(n):
    return sum(i * i for i in range(n))


with ProcessPoolExecutor() as pool:
    results = list(pool.map(heavy, [10_000_000] * 4))

Separate processes, so each gets its own core. The cost is that data must be copied between them, which makes this a poor choice when the work per item is small.

Race conditions#

The first of the two bugs. Two threads read and write the same value, and the result depends on timing:

import threading

counter = 0

def increment():
    global counter
    for _ in range(100_000):
        counter += 1        # read, add, write - three separate steps


threads = [threading.Thread(target=increment) for _ in range(4)]
for t in threads: t.start()
for t in threads: t.join()

print(counter)   # not 400000, and a different number each run

counter += 1 looks like one operation and is three. Two threads can both read 5, both compute 6, and both write 6 — one increment vanishes.

The fix is a lock, so only one thread can be inside the critical section at a time:

lock = threading.Lock()

def increment():
    global counter
    for _ in range(100_000):
        with lock:
            counter += 1

The better fix is usually to avoid shared mutable state altogether — have each thread compute its own total and combine the results at the end. A race condition that cannot occur beats one that is locked against.

Deadlocks#

The second bug. Two threads each hold something the other needs, and neither can continue:

lock_a = threading.Lock()
lock_b = threading.Lock()

def task_one():
    with lock_a:
        with lock_b:      # waits for task_two to release lock_b
            pass

def task_two():
    with lock_b:
        with lock_a:      # waits for task_one to release lock_a
            pass

Neither thread errors. The program simply stops, which makes deadlocks harder to diagnose than crashes.

Two reliable defences: always acquire locks in the same order everywhere in the program, and use a timeout so a stuck thread eventually gives up and reports it.

if lock_a.acquire(timeout=5):
    try:
        ...
    finally:
        lock_a.release()
else:
    raise TimeoutError("Could not acquire lock_a")

How the languages differ#

When not to reach for it#

Concurrency makes programs harder to reason about, harder to debug and harder to test. Bugs appear intermittently and often vanish when you add a print statement. Before adding it, ask:

• Is the program actually too slow? Measure before optimising.
• Is the bottleneck what you think? Profile it.
• Would a better algorithm do more? Often it would, with none of the complexity.
• Is caching an option? Not doing the work at all beats doing it faster.

Where to go next#

• What is an algorithm? — often a better speed-up than concurrency.
• Web scraping and automation — a natural I/O-bound use.
• Handling errors and exceptions — harder, and more important, across threads.