Lock Contention

Lock contention occurs when multiple threads frequently try to acquire the same lock at the same time. Only one wins; the rest block. The protected section becomes a serialization point, and adding cores or threads no longer increases throughput — it may even reduce it as coordination overhead grows.

Why It Happens

A single coarse-grained lock is the easiest way to make shared state thread-safe, and it is correct, so it ships. As traffic grows, that one lock sits on the hot path of every request. Global singletons — a shared counter, a connection pool, an in-memory cache, a logger — are common culprits. Holding a lock across slow operations (I/O, allocation, callbacks) widens the critical section and multiplies contention. Developers rarely measure how much time threads spend waiting versus working.

Why It Hurts

Amdahl's law sets a hard ceiling: if 10% of work is serialized behind a lock, you can never get more than a 10x speedup no matter how many cores you add. In practice it is worse — contended locks cause context switches, cache-line bouncing, and convoy effects where threads queue behind one slow holder. Tail latency explodes because requests randomly stall waiting for the lock. The system looks busy but does little useful work, and the only "fix" appears to be bigger machines that do not help.

Warning Signs

• Throughput stays flat (or drops) as you add threads or cores.
• Thread dumps show many threads BLOCKED on the same monitor.
• Profilers attribute large time to lock acquisition or synchronized blocks.
• A single global mutex guards a frequently accessed structure.
• I/O, logging, or allocation happens while a lock is held.

Better Alternatives

• Shrink the critical section: compute outside the lock, hold it only for the minimal state mutation.
• Partition / shard state: stripe a map into N segments each with its own lock, or use per-thread state aggregated later.
• Read-write locks when reads dominate writes.
• Lock-free / wait-free structures (atomics, CAS-based queues, concurrent maps) for hot counters and queues.
• Immutable data + copy-on-write to avoid locking reads entirely.
• Optimistic concurrency (versioned updates) instead of pessimistic locking.

How to Refactor Out of It

Profile to find the most contended lock — do not guess. Move any I/O, allocation, or callback out of the critical section first; that alone often resolves the problem. Then narrow what the lock protects. If a single structure is the bottleneck, replace it with a sharded or concurrent equivalent so independent keys do not collide. For pure counters and flags, switch to atomics. For read-heavy access, adopt a read-write lock or an immutable snapshot. Re-measure throughput against core count; a healthy system should scale until another resource (CPU, I/O) becomes the limit.