Request Coalescing

When a popular cache entry expires, many concurrent requests can miss simultaneously and all stampede the backend to recompute the same value — a cache stampede (or dogpile). The backend, sized for the cache hit rate, is suddenly hit by a flood of identical, expensive calls. Request Coalescing (also called request collapsing or single-flight) prevents this: when several callers ask for the same thing at the same time, only the first triggers the actual work; the rest wait and share its result.

How It Works

The system keys in-flight requests. When a request arrives, it checks whether an identical request (same key) is already being processed:

• If not, it becomes the leader: it starts the backend call and registers the in-flight operation under the key.
• If yes, it becomes a follower: it attaches to the existing in-flight operation and waits for the leader's result instead of issuing its own call.

When the leader completes, every waiter receives the same result (and value, success or error). The in-flight entry is then removed. Go's singleflight package is a canonical implementation; CDNs (Varnish, NGINX, Cloudflare) call it request collapsing; many caching libraries offer it as stampede protection.

When to Use It

Use coalescing for expensive, cacheable reads with hot keys: rendering a popular page, computing an aggregate, calling a slow upstream, or repopulating a shared cache entry after expiry. It is most valuable where the same value is requested concurrently many times and recomputing it repeatedly is wasteful or harmful.

Trade-offs

Coalescing couples followers to the leader: if the leader's call is slow, all waiters wait, and if it fails, all may receive the same error (so error caching policy matters). It only helps for identical concurrent requests — it does nothing for distinct keys or sequential traffic. In distributed deployments, coalescing is usually per-node, so N nodes can still produce N backend calls unless a shared lock is used, which adds its own latency and complexity. There is also a small risk of serializing throughput on a hot key.

Related Patterns

Request coalescing is a stampede-protection complement to cache-aside, contrasts with hedged-requests (which adds duplicate calls to cut latency rather than removing them), and works alongside rate-limiter and bulkhead-pattern to protect backends.

Example

Go single-flight coalescing:

val, err, _ := group.Do(key, func() (any, error) {
  return backend.expensiveLoad(key)   // runs once per key
})

If a thousand goroutines call Do with the same key while the load is in flight, the backend is hit exactly once and all thousand receive the same result.