Idempotent Writer

An idempotent writer ensures that applying the same write more than once has the same effect as applying it once. In distributed systems, retries and at-least-once message delivery make duplicate operations inevitable: a client times out and retries, a broker redelivers, a consumer restarts mid-batch. Without idempotency, duplicates corrupt state — double charges, double inventory decrements, duplicate rows.

How It Works

The pattern attaches a stable identity to each operation and records which identities have been applied. Common techniques:

• Idempotency key: the client sends a unique key per logical operation; the server stores it with the result and returns the stored result on any repeat, performing the side effect only once.
• Deduplication table / processed-message log: consumers record processed message IDs and skip ones already seen.
• Upsert / conditional write: use INSERT ... ON CONFLICT, MERGE, or compare-and-set so a repeat is a no-op or an identical overwrite.
• Natural idempotency: design operations as set-state ("set status to shipped") rather than deltas ("increment"), so repetition is harmless.
• Monotonic versioning / fencing tokens: reject writes carrying an older version than already applied.

The dedup record and the state change should commit atomically (same transaction) so a crash cannot apply one without the other.

When to Use It

Use idempotent writers anywhere duplicates can occur: message consumers under at-least-once delivery, retried HTTP requests (especially payments and orders), webhook handlers, and asynchronous flush paths in write-behind caches. It is a prerequisite for safe retries in sagas and event-driven systems.

Trade-offs

Tracking applied identities costs storage and a lookup on every write, and the dedup store must be cleaned up or bounded (TTL on keys). Defining the right idempotency scope is subtle: too broad and distinct operations collide; too narrow and real duplicates slip through. Exactly-once semantics are approximated, not guaranteed end-to-end, and atomicity between the dedup record and the side effect is essential to avoid gaps.

Related Patterns

Idempotent writers are foundational to the saga-pattern (compensations and steps must be retry-safe), event-driven-architecture consumers, and the asynchronous flush of a write-behind-cache.

Example

A payment API requires an Idempotency-Key header per charge. On the first request it charges the card, stores the key with the resulting transaction, and returns it. If the client retries after a timeout, the server finds the stored key and returns the original transaction without charging again, so a network hiccup never double-bills the customer.