Messaging

Publish-Subscribe

Decouples senders from receivers by routing messages through a broker or channel so publishers and subscribers never reference each other.

Scatter-Gather

Broadcasts a request to multiple recipients in parallel, then aggregates their replies into a single response.

Choreography

Coordinates a distributed workflow through services reacting to each other's events, with no central controller.

Orchestration

Coordinates a distributed workflow through a central orchestrator that explicitly invokes each service in sequence.

Competing Consumers

Enable multiple concurrent consumers to process messages from the same queue to increase throughput and improve resilience.

Queue-Based Load Leveling

Use a queue between tasks and a service to smooth intermittent heavy loads and protect the service from being overwhelmed.

Claim Check

Store a large message payload externally and pass only a reference through the messaging system to avoid moving bulky data.

Pipes and Filters

Decompose complex processing into a sequence of independent components connected by channels so each step can scale and evolve.

Sequential Convoy

Process related messages in order while still processing unrelated messages in parallel, by grouping them into ordered sets.

Message Channel

Connects two applications with a logical pipe so a sender can transmit data to a receiver without knowing the receiver's location or identity.

Point-to-Point Channel

Ensures exactly one receiver consumes each message on a channel, even when multiple consumers compete, so a message is processed once.

Publish-Subscribe Channel

Broadcasts each message to all interested subscribers so one event can notify many independent consumers without the publisher knowing them.

Message Router

Consumes a message and redirects it to a different channel based on conditions, decoupling producers from the decision of where messages should go.

Content-Based Router

Routes each message to a destination channel chosen by inspecting the message's content, so the payload itself determines where it is delivered.

Message Filter

Lets only messages meeting specified criteria pass through a channel and silently discards the rest, so consumers receive only relevant messages.

Splitter

Breaks a composite message into a series of individual messages so each element can be processed independently downstream.

Message Translator

Converts a message from one data format or schema to another so systems with incompatible representations can communicate.

Content Enricher

Augments a message with additional data from an external source when the original lacks information the receiver requires.

Normalizer

Translates messages arriving in many different formats into a single common format so downstream components handle one canonical representation.

Polling Consumer

A consumer that explicitly checks a channel for messages on its own schedule, controlling exactly when and how fast it receives them.

Event-Driven Consumer

A consumer that is invoked by the messaging system the moment a message arrives, reacting to messages instead of polling for them.

Idempotent Receiver

Makes a consumer safely handle duplicate messages so that processing the same message more than once has the same effect as processing it once.

Dead Letter Channel

Routes messages that cannot be delivered or processed to a dedicated channel for inspection and recovery instead of discarding or blocking them.

Guaranteed Delivery

Persists messages so they are not lost if the sender, broker, or receiver fails, ensuring each message is eventually delivered despite outages.

Request-Reply

Lets a requestor send a message and receive a corresponding response over messaging, combining two one-way channels into a logical two-way exchange.

Correlation Identifier

Tags messages with a unique id that links related messages together, so a reply can be matched to its request and parts to their whole.

Wire Tap

Copies messages flowing through a channel to a secondary channel for inspection, logging, or analysis without disturbing the primary flow.

Routing Slip

Attaches a sequence of processing steps to a message so it routes itself through a series of components determined per message at runtime.

Process Manager

A central component that maintains the state of a multi-step message flow and decides the next step, coordinating complex or branching workflows.

Event-Driven Architecture

An architectural style where components communicate by producing and reacting to events, enabling loose coupling, asynchronous flow, and independent scaling.

Idempotent Writer

A pattern that makes repeated writes safe by ensuring duplicate operations produce the same result, essential for at-least-once delivery and retries.

Idempotency Key

Attaches a unique key to a request so the server can detect and de-duplicate retries, making non-idempotent operations safe to repeat.

Anti-Patterns1

Anti-Pattern

Missing Back-Pressure

A producer that pushes work faster than the consumer can handle, with no mechanism to slow it down, so queues grow unbounded until memory or the downstream collapses.

Tutorials3

Tutorial

Building Event-Driven Systems with Kafka

Create scalable event-driven architectures using Apache Kafka

Tutorial

Real-time Communication with WebSockets

Implement real-time features using WebSockets

Tutorial

How to build a Kafka producer and consumer

Create a Kafka topic, produce messages with keys, and consume them in a group, covering offsets, partitions, and delivery basics.

Blueprints10

Scala Akka Actor Modernization Blueprint

Modernize legacy Scala Akka classic actor systems to typed actors and current streaming APIs while addressing licensing changes.

Synchronous REST to Event-Driven Blueprint

Convert tightly-coupled synchronous REST call chains into an event-driven architecture using a broker, async messaging, and the outbox pattern.

Batch to Streaming with Apache Kafka Blueprint

Replace nightly batch ETL with real-time event streaming on Apache Kafka for low-latency data movement.

Build a Change Data Capture (CDC) Pipeline Blueprint

Stand up a CDC pipeline with Debezium and Kafka to stream database changes in real time to downstream consumers.

Synchronous Calls to Event-Driven (Kafka) Blueprint

Replace brittle synchronous service-to-service calls with asynchronous events on Apache Kafka to decouple services and improve resilience.

Synchronous API to SNS/SQS Fan-Out Blueprint

Decouple synchronous workloads using Amazon SNS topics and SQS queues for durable, buffered fan-out and asynchronous processing.

Point-to-Point Integrations to Message Bus Blueprint

Replace tangled point-to-point integrations with a central message bus and publish/subscribe model to reduce coupling and integration sprawl.

Webhooks to Event Streaming Blueprint

Evolve unreliable HTTP webhook integrations into a durable event-streaming backbone with replay, ordering, and at-least-once delivery.

Central Orchestration to Choreography (Saga) Blueprint

Replace a central orchestrator coordinating distributed transactions with event-driven choreography using the saga pattern and compensations.

Batch File Transfer to Streaming CDC Blueprint

Replace nightly batch file transfers between systems with real-time change data capture streamed onto an event backbone.

Products6

Apache Kafka

Distributed event streaming platform

Apache Pulsar

Cloud-native distributed messaging and streaming

Amazon SQS

Fully managed message queuing service

Reference Architectures11

Event-Driven Microservices on Kubernetes

A Kubernetes-native reference design for loosely coupled microservices that communicate through Kafka events with service-level autoscaling.

CQRS and Event Sourcing on Cloud

A cloud microservices design separating write and read models with event sourcing for full auditability and independent scaling.

gRPC Service Mesh for Internal APIs

A high-performance internal API platform using gRPC over a service mesh for typed, low-latency service-to-service calls.

Event-Driven Backbone with Kafka

An organization-wide event streaming backbone on Kafka that decouples producers and consumers through durable, replayable topics.

Reliable Webhook Delivery Platform

A platform that delivers outbound webhooks to customer endpoints with retries, signing, idempotency, and per-tenant rate control.

Pub/Sub Fan-Out for Event Distribution

A publish-subscribe fan-out architecture that broadcasts each event to many independent consumers with per-subscriber queues.

Real-Time Notification Delivery System

A real-time notification system that pushes in-app, push, email, and SMS messages to users with preference and channel routing.

Event Choreography for Microservices

A choreographed event-driven design where services react to each other's domain events without a central orchestrator.

Asynchronous Task Queue with Workers

A durable task queue that offloads slow or unreliable work from request handlers to scalable background workers with retries.

Transactional Outbox for Reliable Events

A transactional outbox design that guarantees events are published exactly when their database changes commit, avoiding dual-write loss.

IoT Ingestion Platform

A platform that ingests telemetry from large device fleets over MQTT, processes it as a stream, and stores it for analytics and control.

Playbooks7

Event-Driven Architecture Adoption Program Playbook

A program to introduce event-driven communication, a schema registry, and async workflows into a synchronous backend estate.

Batch to Streaming Program Playbook

Transform nightly batch data pipelines into real-time streaming pipelines using event-driven architecture and stream processing.

CDC and Real-Time Pipeline Program Playbook

Build change data capture pipelines to stream database changes into analytics and event systems with low latency and strong consistency.

Event-Driven Architecture Adoption Playbook

A program to adopt event-driven architecture with a streaming backbone, schema governance, and resilient async patterns.

ESB to Modern Integration Playbook

A program to migrate from a centralized enterprise service bus to decentralized, event-driven and API-led integration.

Webhook Platform Rollout Playbook

A program to build a reliable webhook delivery platform with signing, retries, idempotency, and subscriber management.