Skip to main content

Central Orchestration to Choreography (Saga) Blueprint

Replace a bottleneck orchestrator with event-driven choreography using the saga pattern, where each service reacts to events and failures unwind via idempotent compensations. Tracing and a saga-state projection restore the visibility choreography hides.

From
Orchestration
To
Choreography
Difficulty
Expert
Duration
18 weeks
Team Size
large

What and Why

A central orchestrator that drives a multi-service business transaction becomes a bottleneck and a coupling point: it must know every service and every step. Choreography distributes the workflow as events: each service reacts to events and emits new ones, and the saga pattern coordinates a distributed transaction through a sequence of local transactions with compensating actions on failure.

This blueprint moves a workflow from orchestrated control to event choreography where appropriate, keeping orchestration only where central visibility is essential.

Phases

Assessment. Map the orchestrated workflow, its steps, failure points, and which transactions need rollback. Decide step by step whether choreography or orchestration suits each (long, visibility-heavy flows may stay orchestrated).

Saga design. Model the saga: the sequence of local transactions, the events between them, and a compensating action for each step. Define success and failure paths explicitly.

Event choreography. Implement services that subscribe to and publish domain events on Kafka. Each service owns its data (database-per-service) and reacts autonomously. Use CQRS where read and write models differ.

Compensation logic. Implement idempotent compensating transactions so a failed saga unwinds prior steps consistently. Handle retries and timeouts.

Migration. Run the choreographed path beside the orchestrator, comparing outcomes, then retire the orchestrator's role step by step.

Key Risks and Mitigations

  • Partial failure: a saga can fail midway. Implement and test idempotent compensations for every step, with timeouts and retries.
  • Compensation complexity: unwinding distributed state is hard to reason about. Keep sagas short, document state transitions, and consider a lightweight saga library or stateful stream processor for tracking.
  • Loss of visibility: choreography hides the end-to-end flow. Add distributed tracing and a saga-state projection so operators can see in-flight transactions.

Recommended Tooling

Kafka with a schema registry for events, a saga framework or Kafka Streams for state tracking, database-per-service stores (PostgreSQL), OpenTelemetry for tracing the distributed flow, and dead-letter handling for stuck sagas.

Success Metrics

Measure coupling reduction (the orchestrator's dependencies removed), resilience to single-service failures, throughput, and successful compensation rate under induced failures.

Prerequisites

Clear domain boundaries (database-per-service), an event-streaming platform, distributed tracing, and team maturity with eventual consistency and idempotency.