On-Call Best Practices

Best Practice: On-Call Best Practices

On-call is the practice of having engineers available to respond to production issues outside normal hours. Done well, it keeps services reliable while keeping people healthy. Done badly, it produces alert fatigue, burnout, and slow responses. The core disciplines are humane rotation design, clear escalation, alerts that are always actionable, and good runbooks so any responder can act.

It matters because reliability depends on people, and people degrade when paged constantly or at random. Treating on-call load as a measurable, managed thing, with limits on pages per shift and follow-the-sun rotations where possible, makes both the service and the team sustainable, which is essential through long migration programs.

Google SRE offers a useful guideline: on-call engineers should spend no more than about half their time on operational work, and a shift should not exceed roughly two significant incidents, so there is time to handle each one properly and still recover. Excessive paging is a signal that the system or the alerting needs engineering attention, not that the team needs to try harder. The people who build a service should also operate it, because that feedback loop motivates them to make it reliable and easy to run. On-call also needs psychological safety: clear escalation means no one is stuck alone, and the expectation that you can wake a secondary or a subject-matter expert without shame. Treating on-call as a first-class, compensated responsibility rather than an unspoken tax is what keeps experienced engineers willing to do it.

Step-by-Step Implementation Guidance

Build rotations with enough people that no one is on-call too often; aim for reasonable shift length.
Define primary and secondary on-call plus a clear escalation path to subject-matter experts.
Ensure every alert that pages is actionable and links to a runbook.
Track page volume and noise; treat excessive paging as a bug to fix, not a fact of life.
Use follow-the-sun rotations across regions to avoid night paging where staffing allows.
Compensate and recognize on-call work explicitly.
Hand off cleanly between shifts and review on-call health regularly.

Common Mistakes Teams Make When Ignoring This Practice

One or two heroes carrying all on-call, risking burnout and bus factor.
Pages that are not actionable, training responders to ignore them.
No escalation path, leaving a stuck responder alone at 3 a.m.
Ignoring rising page volume until people quit.
No runbooks, so only experts can resolve anything.

Tools and Techniques That Support This Practice

PagerDuty and Opsgenie for scheduling, escalation, and analytics.
Runbook repositories linked from alerts.
Page-volume and time-to-acknowledge dashboards.
Chatops for coordinated response and handoff.

How This Practice Applies to Different Migration Types

Cloud Migration: Staff extra on-call coverage and clear escalation around cutover windows.
Database Migration: Ensure DBAs are in the escalation path during data migration shifts.
SaaS Migration: Document vendor support contacts in escalation so responders are not blocked.
Codebase Migration: Provide runbooks for the new system so on-call can support it from day one.

Checklist

Rotation sized so no one is overloaded
Primary, secondary, and escalation path defined
Every page is actionable with a runbook
Page volume and noise tracked
Follow-the-sun coverage used where possible
On-call work compensated and recognized
Clean shift handoff process in place