Cache Stampede (Dogpile Effect)

A cache stampede (also called the dogpile or thundering-herd-on-cache effect) happens when a popular cached value expires and, in the brief window before it is repopulated, every concurrent request misses the cache simultaneously and tries to recompute the same value. Instead of one recomputation, the backing database or service is hit by hundreds at once — a sudden, correlated load spike triggered by a single key's expiry.

Why It Happens

The naive cache pattern is read-through: on a miss, compute the value and store it. This is correct for one request but has no coordination across concurrent requests. When a high-traffic key expires, all in-flight requests see a miss before any of them finishes recomputing and writing back. Setting the same TTL on many keys makes it worse by synchronizing their expirations. The problem is invisible until a key is both popular and expensive to compute.

Why It Hurts

The whole point of caching is to shield the backend, but a stampede does the opposite: at the moment of expiry it directs the full unbuffered load at the slowest component. The expensive computation runs N times in parallel, each consuming connections and CPU, which can overload the database, spike latency for everyone, and cascade into timeouts and retry storms that prolong the outage. Because it is tied to TTL boundaries, it recurs on a schedule.

Warning Signs

• Backend load spikes that line up with cache TTL boundaries.
• Bursts of identical, simultaneous recomputations of the same key.
• Database CPU or latency periodically jumping then settling.
• Many keys created together expiring together.

Better Alternatives

• Request coalescing / single-flight: let only one request recompute a missing key while others wait for and share its result.
• Probabilistic early expiration (XFetch): refresh slightly before expiry, staggered per request, so recomputation happens before a hard miss.
• Stale-while-revalidate: serve the stale value immediately and refresh asynchronously in the background.
• Locks/leases on recompute: a short-lived lock per key ensures one recomputation at a time.
• TTL jitter: randomize expirations so keys do not expire in sync.

How to Refactor Out of It

Wrap cache misses in a single-flight mechanism so concurrent callers for the same key share one computation rather than each running it. For hot keys, add stale-while-revalidate so users get the cached value instantly while a background task refreshes it, eliminating the miss window entirely. Apply probabilistic early refresh and add jitter to TTLs so expirations spread out instead of clustering. Load-test by expiring a hot key under concurrency and confirm the backend sees one recompute, not a herd.