Flaky Tests

Flaky tests fail intermittently for the same code, sometimes passing on a re-run with no changes. They are among the most corrosive problems in a test suite because they teach the team to distrust red builds.

Why It Happens

Flakiness usually comes from non-determinism. Common sources include reliance on real time and sleep calls instead of synchronization, hidden dependencies on test execution order, shared mutable state between tests, race conditions in async code, dependence on external services or networks, and unstable assertions such as comparing unordered collections as if ordered. UI tests are especially prone because they assume elements render within a fixed delay.

Why It Hurts

When a build is sometimes red for no real reason, developers start re-running until green or adding blanket retries. That habit also hides genuine regressions: a real failure looks identical to a flaky one and gets re-run away. CI becomes slower and more expensive as jobs retry. Eventually teams disable or ignore whole suites, losing the protection they paid for. Flakiness is therefore not a minor annoyance — it directly undermines the value of automated testing.

Warning Signs

• The team's reflex for a red build is "just re-run it."
• Tests pass locally but fail in CI, or vice versa.
• Failures cluster around timing, dates, or concurrency.
• A list of "known flaky" tests is tolerated indefinitely.

Better Alternatives

Make tests deterministic. Control time with a clock abstraction or fake timers instead of sleeping. Isolate tests so each sets up and tears down its own state and can run in any order or in parallel. Replace real external calls with controlled test doubles or hermetic test environments. Wait on conditions (poll until ready) rather than fixed delays. Treat a flaky test as a defect with an owner, not background noise.

How to Refactor Out of It

First, detect and quantify: run suites repeatedly or use CI analytics to identify which tests flake and how often. Quarantine the worst offenders into a separate, non-blocking lane so they stop polluting signal — but track them, because quarantine is a holding pen, not a graveyard. Fix root causes one at a time: remove order dependencies, inject clocks, stabilize selectors, and eliminate shared state. Add guardrails so new flakiness is caught early, such as running the suite multiple times on a schedule. Set a policy that a test flaky beyond a threshold is fixed or deleted within a fixed window. A green build must mean the code is healthy — nothing less.