Retry

The Retry pattern handles transient failures by automatically reattempting a failed operation, expecting that the fault is temporary and will clear on a subsequent try. In cloud systems, brief network glitches, momentary service unavailability, and throttling responses are common and self-correcting, so a well-judged retry restores success without surfacing an error.

The problem is that distributed systems fail in small, temporary ways constantly. Treating every transient blip as a hard failure makes systems fragile and degrades user experience.

How It Works

When an operation fails, the caller inspects the error to decide whether it is transient (timeout, connection reset, HTTP 503 or 429) or permanent (authentication failure, bad request). For transient errors, it waits and retries, up to a maximum number of attempts.

The delay strategy matters. Fixed delay retries at constant intervals. Exponential backoff increases the wait after each attempt, easing pressure on a struggling service. Jitter adds randomness to the delay so many clients do not retry in lockstep and cause a thundering herd. After the attempt limit, the operation fails and the error propagates.

Operations being retried should be idempotent, because a request may have succeeded before the response was lost, and retrying could duplicate effects.

When to Use It

Use it for operations subject to transient faults: remote service calls, database connections, and cloud API requests, especially idempotent ones. Use exponential backoff with jitter for shared or rate-limited services.

Avoid retrying non-transient errors (they will not succeed and waste time), and avoid retrying non-idempotent operations without safeguards like idempotency keys.

Trade-offs

Retries add latency and load; aggressive retrying against a struggling service can worsen its overload, a retry storm. Limits, backoff, and jitter are essential to avoid amplifying failures. Without idempotency, retries risk duplicate side effects. Distinguishing transient from permanent faults is not always clear-cut.

Done well, retry dramatically improves perceived reliability by absorbing the routine transient faults of distributed systems.

Related Patterns

It is almost always paired with Circuit Breaker, which stops retries when a dependency is persistently down, preventing retry storms. Throttling and Rate Limiting bound the load that retries generate. Together these form a resilience toolkit.

Example

A service calls a payment API. A request times out, a transient fault. The client retries after 200 ms, then 400 ms, then 800 ms, each with random jitter, up to three attempts. The second attempt succeeds, and the user never sees an error. The request carries an idempotency key so that even if the original attempt actually charged the card before timing out, the retry does not double-charge.