Parallelism

Parallelism is the technique of running multiple computations at the same instant, using multiple processing units to finish a workload faster. Where concurrency is about structuring tasks so they can overlap, parallelism is about actually executing them simultaneously on parallel hardware.

How It Works

Parallelism requires more than one execution resource: multiple CPU cores, multiple GPUs, or multiple machines. A problem is divided into pieces that run at once, then results are combined. Common forms include data parallelism, where the same operation runs over different chunks of data (as in GPU computing or map-reduce), and task parallelism, where different operations run concurrently. Effective parallelism depends on minimizing coordination overhead and dependencies between pieces. Amdahl's law captures a key limit: the speedup is bounded by the fraction of work that must remain sequential.

Why It Matters

Processor clock speeds stopped rising sharply years ago, so performance gains now come largely from using more cores. Parallelism is how software turns multi-core CPUs, GPUs, and clusters into faster results — for data processing, machine-learning training, scientific computing, and rendering. Without it, much modern hardware sits idle.

Parallelism shares concurrency's hazards around shared mutable state, and adds the challenge of partitioning work evenly. Approaches that avoid shared state — immutable data, pure functions, message passing — scale far better in parallel settings.

Related Terms

Concurrency structures overlapping tasks and is often confused with parallelism. Immutability and pure functions make parallel code safe and scalable. Performance is the goal parallelism serves.