Scatter-Gather

Scatter-Gather is an enterprise integration pattern that sends a request to many recipients at once (scatter) and then collects and combines their responses (gather). It is used when an answer requires input from multiple parties, or when you want to ask several providers and pick the best reply, all in parallel to keep latency low.

How It Works

A scatterer broadcasts the request to a set of recipients, either a known list or all subscribers to a channel. Each recipient processes the request independently and returns a response. A gatherer collects these responses, correlating them to the original request, and aggregates them, by merging, selecting the best, or summarizing, into one result for the caller.

Because not all recipients may reply, the gatherer applies a strategy: wait for all, wait for a quorum, or wait until a timeout and return whatever arrived. Correlation identifiers tie responses back to the request.

Two common topologies exist: a distribution list, where the scatterer knows each recipient and sends directly, and a publish-subscribe channel, where any number of interested recipients respond. The gatherer relies on a correlation identifier to match replies to the original request and an aggregation strategy, first, fastest, quorum, or all, to decide when it has enough to respond.

When to Use It

Use scatter-gather when a result needs contributions from multiple services or when you solicit competing offers and choose among them, such as querying several suppliers for a price. It is also the standard way to run a query across data shards and merge the partial results. Parallel fan-out keeps total latency near that of the slowest needed responder rather than the sum.

If only one recipient holds the answer, a direct call is simpler.

It is also well suited to redundancy strategies that send the same request to several equivalent providers and take the first valid answer, trading extra load for lower tail latency.

Trade-offs

Scatter-gather parallelizes work but its latency is bounded by the slowest responder you wait for, so timeout and partial-result policies are essential. Handling missing or duplicate replies adds complexity. Broadcasting can put load on many recipients. The aggregation logic must correlate responses correctly. The benefit is fast, comprehensive results from many sources. Setting a sensible aggregation policy up front, deciding exactly how long to wait and how many replies constitute success, is what keeps the pattern fast and predictable instead of hostage to one slow recipient.

Related Patterns

Scatter-Gather is the messaging form of the Aggregator pattern; API Composition and Gateway Aggregation are request/response variants. It is the standard query mechanism across Sharding partitions. Sharding defines the partitions it queries, and the Aggregator captures the same idea for synchronous request/response.

Example

A travel site quotes the cheapest flight by asking five airline services simultaneously. It scatters the search to all five, then gathers replies for up to 800 ms. It selects the lowest fare among whatever responses arrived; if one airline is slow, its quote is simply excluded. The user gets a fast result drawn from multiple providers.