Thread Pool

A thread pool maintains a set of pre-created worker threads that pull tasks from a shared queue and execute them. Instead of spawning a new thread per task — which is expensive and unbounded — work is submitted to the pool, which reuses its threads. The pool caps concurrency, amortizes thread-creation cost, and provides a single place to manage execution policy.

How It Works

The pool combines a task queue with a fixed or elastic group of worker threads (a producer-consumer arrangement). Callers submit tasks (often returning a future or promise for the result); idle workers dequeue and run them. Key policies include pool size, queue type and bound, rejection behavior when the queue is full, and keep-alive for elastic pools. Standard libraries provide this directly: Java's ExecutorService/ThreadPoolExecutor, .NET's ThreadPool and TPL, Python's ThreadPoolExecutor, and language runtimes' own schedulers. Sizing typically targets the number of CPU cores for CPU-bound work and a higher count for I/O-bound work that spends time waiting.

When to Use It

Use a thread pool for workloads with many short-lived, independent tasks: handling server requests, running background jobs, parallelizing batch work, and executing callbacks. It is the default execution substrate for servers and the natural home for the consumer side of producer-consumer.

Trade-offs

A pool sized too small underutilizes the machine and queues work; too large causes context-switching overhead and resource contention. Mixing blocking I/O into a CPU-sized pool can starve it (thread-pool starvation), and long-running or blocking tasks can monopolize workers. Tasks must not depend on each other in ways that deadlock the pool (a task waiting on another task in the same exhausted pool). Exception handling, naming, and graceful shutdown need explicit attention.

Related Patterns

A thread pool is a concrete realization of producer-consumer (workers consume a task queue), pairs with fork-join for divide-and-conquer parallelism, and contrasts with the leader-follower model of distributing work without a central dispatcher.

Example

An HTTP server submits each incoming request to a fixed thread pool of sixty-four workers backed by a bounded queue. Under a traffic spike the queue absorbs bursts and excess requests are rejected with a 503 rather than spawning unbounded threads, keeping the server responsive and its memory bounded.