Rate Limiting
Rate Limiting constrains the rate of operations using algorithms like token bucket so callers stay within quotas and avoid throttling, errors, and overload. It smooths bursts and protects shared resources at the cost of added latency and shared-state coordination.
The Rate Limiting pattern controls how frequently operations are issued so they stay within an allowed rate. While throttling usually protects a service from inbound load, rate limiting often describes a client constraining its own outbound calls to respect a downstream service's quotas and avoid being throttled or hitting errors.
The problem is that services and APIs enforce rate quotas. Exceeding them causes rejected requests (HTTP 429), errors, or extra cost. Uncontrolled bursts also overwhelm shared resources.
How It Works
Rate limiting uses algorithms to decide whether an operation may proceed now. The token bucket algorithm refills tokens at a steady rate up to a burst capacity; each operation consumes a token, and operations wait or are rejected when none remain. The leaky bucket smooths output to a constant rate. Fixed-window and sliding-window counters cap operations per time interval.
A limiter can block (wait until allowed), queue, or reject excess operations. In distributed systems, the limiter state lives in a shared store (such as Redis) so a global limit is enforced across many instances. Clients also read response headers to adapt to a server's advertised limits.
When to Use It
Use it when calling APIs or resources with quotas (payment gateways, cloud provider APIs, third-party services), to avoid self-induced throttling and errors, and to smooth bursts into a steady rate. Use it to enforce fair use among internal callers of a shared dependency.
Avoid unnecessary limiting on cheap, unbounded local operations where it only adds latency.
Trade-offs
Rate limiting adds latency when it makes operations wait, and rejected operations need handling, typically retry with backoff. Choosing the right algorithm and limits requires knowing the target's quotas and traffic shape. Distributed limiting needs shared state, adding a coordination point and latency. Too strict a limit underuses capacity; too loose defeats the purpose.
The benefit is staying within quotas, avoiding errors and extra cost, and protecting shared resources.
Related Patterns
It is the client-side, algorithmic counterpart to Throttling. It works with Retry (with exponential backoff and jitter) to handle rejected calls and with Queue-Based Load Leveling to defer excess work rather than reject it.
Example
A service syncs data to a third-party API capped at 10 requests per second. The service uses a token-bucket limiter backed by Redis so all instances share one global rate of 10 per second with a small burst allowance. When the bucket empties, calls wait briefly rather than firing and getting 429s. Combined with retry-with-backoff for the rare rejection, the integration runs at maximum allowed throughput without tripping the provider's limits.