Idempotency Key

Networks are unreliable, so clients retry. But retrying a non-idempotent operation — charge a card, create an order, send a message — risks doing it twice if the original actually succeeded but the response was lost. The Idempotency Key pattern solves this: the client generates a unique key for each logical operation and sends it with the request. The server records the key and its result, so any retry carrying the same key returns the original outcome instead of executing the operation again.

How It Works

1. The client creates a unique key (typically a UUID) for one logical request and includes it, often as an Idempotency-Key header.
2. On first receipt, the server checks whether the key exists. It does not, so the server processes the request, then stores the key with the response (and status).
3. On a retry with the same key, the server finds the stored record and returns the saved response without re-executing — no duplicate charge or order.

Key design concerns: the store must be durable and consistent (a database row with a unique constraint, or Redis with care), in-flight requests need locking to avoid a race where two retries process concurrently, keys have a retention window (commonly 24 hours to a few days), and the same key must map to the same request payload — many APIs reject a key reused with a different body.

When to Use It

Use idempotency keys for any write operation exposed over an unreliable channel that clients may retry: payment and billing APIs, order creation, account provisioning, and any POST that has side effects. They are equally vital for consumers of at-least-once messaging, where the same message may be delivered more than once. Stripe's payments API popularized the Idempotency-Key convention.

Trade-offs

The pattern requires durable storage and careful concurrency handling — a naive implementation has a window where two simultaneous retries both execute. It adds latency (a lookup per request) and operational overhead (key expiry, storage growth). Defining the right scope of a "logical operation" and tying the key to the payload requires discipline. None of this is free, but the alternative — duplicate financial side effects — is far costlier.

Related Patterns

Idempotency keys make retry-with-backoff safe, support reliable dead-letter-queue and at-least-once messaging, and underpin recovery in the saga-pattern and outbox-pattern, where steps may be replayed.

Example

Server-side handling:

key = request.header("Idempotency-Key")
rec = store.get(key)
if rec: return rec.response          // retry => replay original
with lock(key):
  resp = process(request)
  store.put(key, resp, ttl=24h)
  return resp

A lost response followed by a client retry yields the same charge exactly once.