Dark Launching

Best Practice: Dark Launching

Dark launching is the practice of deploying new functionality to production but keeping it hidden from users, while exercising it with real or mirrored traffic to validate correctness and capacity. The term comes from Facebook's engineering team, which dark-launched features to test backend load before any user could see them. By separating deploy from release, dark launching lets teams de-risk a launch under real conditions, catch performance and integration problems early, and turn the feature on with confidence. For leaders, it reduces the chance that a high-profile launch fails under real load.

Step-by-Step Implementation Guidance

1. Deploy the new code path to production behind a flag that keeps it invisible to users.
2. Send real or mirrored traffic to the hidden path without surfacing results to users.
3. Measure behavior and load to confirm correctness, latency, and capacity at scale.
4. Compare outputs of the new path against the existing one where applicable.
5. Fix issues found while users remain unaffected by the hidden path.
6. Gradually expose the feature via the flag once it is proven healthy.
7. Remove the dark-launch scaffolding after the feature is fully released.

Common Mistakes Teams Make When Ignoring This Practice

• Confusing dark launching with a feature flag toggle and skipping real-traffic load testing.
• Letting mirrored traffic cause side effects, such as duplicate writes or emails.
• Not measuring capacity, so the real launch still overwhelms the system.
• Leaving dark-launch code paths in place indefinitely, accruing complexity.
• Exposing the feature before the hidden phase has shown it is healthy.

Tools and Techniques That Support This Practice

• Feature flags: LaunchDarkly, Unleash, and Flagsmith to gate visibility.
• Traffic mirroring: Istio, Envoy, and Diffy for shadow traffic and response comparison.
• Observability: Prometheus, Grafana, and APM tools to measure the hidden path.
• Load validation: k6 or Locust to complement real-traffic shadowing.

How This Practice Applies to Different Migration Types

• Cloud Migration: Shadow real traffic to newly migrated cloud services to validate capacity before cutover.
• Database Migration: Dual-run reads against a migrated database in the dark to compare results before switching.
• SaaS Migration: Exercise new SaaS integrations with mirrored traffic before exposing them to customers.
• Codebase Migration: Run a rewritten code path in the dark alongside the old one to compare behavior at scale.

Checklist

• New functionality deploys hidden behind a flag.
• Real or mirrored traffic exercises the path.
• Behavior and capacity are measured at scale.
• Mirrored traffic causes no user-visible side effects.
• The feature is exposed only after proving healthy.
• Dark-launch scaffolding is removed after release.

During modernization and migration work, dark launching is most valuable when it is treated as a continuous discipline rather than a one-time setup. Teams that codify the practice, measure its outcomes, and review it regularly keep risk low and feedback fast as systems evolve. Start small with one team or service, prove the value with concrete metrics such as lead time and change failure rate, and then expand the practice across the portfolio. Pair it with the related practices in this library so that build, test, release, and operational concerns reinforce one another. Documenting decisions and automating enforcement makes the practice durable as people and priorities change, which is exactly what large, multi-team migrations demand to stay safe and predictable over time.