Once you have established that your problem genuinely needs concurrency, the next question is which tool to reach for. This guide is the practical companion to the conceptual one: thread pools instead of raw threads, the synchronisation primitives worth knowing, thread-safe collections, and the patterns that avoid needing any of them.
Do not create threads directly#
import threading
# Works, but creates and destroys a thread per item
for url in urls:
threading.Thread(target=fetch, args=(url,)).start()
Creating a thread is expensive, and a thousand URLs means a thousand threads competing for the processor. A pool creates a fixed number and feeds them work from a queue:
from concurrent.futures import ThreadPoolExecutor
with ThreadPoolExecutor(max_workers=10) as pool:
results = list(pool.map(fetch, urls))
The with block waits for everything to finish and shuts the pool down, including when an exception is raised.
// Java
try (var pool = Executors.newFixedThreadPool(10)) {
List<Future<String>> futures = urls.stream()
.map(url -> pool.submit(() -> fetch(url)))
.toList();
for (Future<String> f : futures) {
System.out.println(f.get());
}
}
Handling results and failures#
from concurrent.futures import ThreadPoolExecutor, as_completed
with ThreadPoolExecutor(max_workers=10) as pool:
futures = {pool.submit(fetch, url): url for url in urls}
for future in as_completed(futures):
url = futures[future]
try:
print(url, "->", len(future.result()))
except Exception as error:
print(url, "failed:", error)
as_completed yields results as they finish rather than in submission order, so slow items do not hold up fast ones.
Sizing the pool#
- CPU-bound work: roughly the number of cores. More threads only add switching overhead.
- I/O-bound work: many more than the core count, since most threads are waiting. Ten to a few hundred, depending on what you are waiting for.
- Mixed: use two pools, so a burst of slow I/O cannot starve your computation.
import os
cpu_pool = ThreadPoolExecutor(max_workers=os.cpu_count())
io_pool = ThreadPoolExecutor(max_workers=32)
These are starting points, not answers. Measure with your real workload.
Locks#
import threading
lock = threading.Lock()
total = 0
def add(amount):
global total
with lock: # only one thread inside at a time
total += amount
Keep the locked region as small as possible. Everything inside it is single-threaded, so a lock held around slow work destroys the benefit of having threads at all.
# Bad - the network call is inside the lock
with lock:
data = fetch(url)
cache[url] = data
# Better - only the shared write is locked
data = fetch(url)
with lock:
cache[url] = data
Other primitives worth knowing:
| Tool | Use for |
|---|---|
Lock |
one thread at a time |
RLock |
the same thread re-entering |
Semaphore |
limiting concurrent access to N |
Event |
signalling that something happened |
Condition |
waiting until a state changes |
Barrier |
waiting for everyone to arrive |
Atomics and thread-safe collections#
Often you can avoid a lock entirely by using something already safe:
// Java
AtomicInteger counter = new AtomicInteger();
counter.incrementAndGet(); // no lock needed
ConcurrentHashMap<String, Integer> map = new ConcurrentHashMap<>();
map.merge("key", 1, Integer::sum);
CopyOnWriteArrayList<String> listeners = new CopyOnWriteArrayList<>();
# Python - queue is thread-safe by design
from queue import Queue
results = Queue()
results.put(value) # no lock needed
CopyOnWriteArrayList copies the whole array on every write, so it suits collections that are read constantly and written rarely — event listener lists are the classic case.
Producer and consumer#
The pattern that removes most locking, because the queue does the synchronising:
import threading
from queue import Queue
work = Queue(maxsize=100)
SENTINEL = object()
def producer():
for item in read_items():
work.put(item) # blocks when the queue is full
for _ in range(4):
work.put(SENTINEL)
def consumer():
while True:
item = work.get()
if item is SENTINEL:
work.task_done()
break
try:
process(item)
finally:
work.task_done()
threads = [threading.Thread(target=consumer) for _ in range(4)]
for t in threads:
t.start()
producer()
for t in threads:
t.join()
The maxsize is doing real work: it applies back pressure, so a fast producer cannot fill memory faster than the consumers can drain it.
Cancellation and timeouts#
stop = threading.Event()
def worker():
while not stop.is_set():
do_a_chunk()
stop.wait(timeout=1) # sleeps, but wakes immediately on stop
stop.set() # ask every worker to finish
There is no safe way to kill a thread from outside in either Python or Java. Cooperative cancellation — a flag the worker checks — is the only correct approach.
future = pool.submit(slow_task)
try:
result = future.result(timeout=5)
except TimeoutError:
print("still running - the task itself is not cancelled")
The patterns that avoid all of this#
- Give each thread its own data and combine at the end. No shared state, no locks, no races.
- Use immutable objects. Anything that cannot change is safe to share.
- Pass messages instead of sharing memory — a queue between stages rather than a shared structure.
- Confine mutable state to one thread and have others send it requests.
from concurrent.futures import ThreadPoolExecutor
# Each task returns its own result - nothing shared
with ThreadPoolExecutor(max_workers=8) as pool:
counts = list(pool.map(count_words_in, files))
total = sum(counts) # combined on one thread
Questions people ask#
How many threads is too many?
Each carries a memory cost for its stack, typically around a megabyte. Thousands of threads means gigabytes and heavy switching. For high concurrency, use async or virtual threads instead.
What are virtual threads?
Lightweight threads managed by the Java runtime rather than the operating system. They cost very little, so you can have millions, which makes the thread-per-request model viable again for I/O-heavy servers.
Do I need a lock to read a shared value?
Usually yes, unless the type is explicitly atomic. Without one you may read a value being written, or a stale cached copy. Use an atomic type or a volatile field.
Why does my concurrent code behave differently on another machine?
Different core counts and timing change how threads interleave. A race condition that never triggers on a laptop can trigger constantly on a server, which is why correctness cannot be established by testing alone.
Where to go next#
- Concurrency and multithreading — the concepts behind these tools.
- Advanced data structures — queues and their thread-safe variants.
- Java collections framework — the concurrent collections in detail.
