Hedged Requests
Hedged Requests cut tail latency by sending a duplicate request to another replica after a short delay and taking the first response. Firing only on the slow tail keeps extra load small, but the technique needs idempotency and bounding to avoid amplifying systemic overload.
In large systems, the slowest few percent of requests dominate user-perceived latency — a phenomenon Google's Jeff Dean and Luiz Barroso analyzed in "The Tail at Scale." A single request that hits a server doing garbage collection, a cold cache, or a noisy-neighbor spike can be far slower than the median, even when the service is healthy overall. Hedged Requests attack tail latency: if a request has not responded within a short delay, the client sends a second copy to another replica and uses whichever answer comes back first, cancelling the loser.
How It Works
The client issues the primary request and starts a timer set to a high percentile of normal latency (for example p95). If the primary responds before the timer fires, nothing extra happens. If the timer fires first, the client dispatches a hedge to a different replica. The first response to arrive wins; the other is cancelled if possible.
Because the hedge fires only for the slow tail, extra load is small — typically a few percent of requests are ever duplicated. A related variant, tied requests, sends both copies immediately but lets the servers coordinate so that once one starts processing, it cancels the other, trimming the tail further at slightly higher cost.
When to Use It
Hedging fits read-heavy, idempotent operations against replicated backends where tail latency matters: distributed databases, storage systems, search, and key-value lookups. It is most valuable when latency SLAs are tight and the backend has multiple equivalent replicas to choose from. gRPC and several distributed databases (Cassandra, DynamoDB clients, BigTable) support hedging directly.
Trade-offs
Hedging trades extra load for lower latency, so it must be bounded — fire only on the slow tail and cap the number of hedges, or it can amplify load during a broad slowdown exactly when the system is fragile. It generally requires idempotent operations, since the same request may be processed twice; writes need care or coordination. Cancellation is best-effort, so wasted work is possible. Hedging also assumes slowness is server-specific (a replica having a bad moment), not a systemic overload that every replica shares.
Related Patterns
Hedged requests complement request-coalescing (deduplicating identical concurrent requests), timeout-pattern (bounding each attempt), and contrast with retry-with-backoff (which waits for failure rather than acting on slowness). load-shedding guards against hedging amplifying systemic overload.
Example
Client-side hedging:
primary = send(replicaA)
result = await(primary, timeout=p95_latency)
if result.timedOut:
hedge = send(replicaB)
result = awaitFirst(primary, hedge) // first wins
cancel(other)
return result
Only genuinely slow requests pay the cost of a second call, while the p99 latency drops sharply.