How to Define SLOs and Error Budgets for a Service

What and why

A Service Level Objective (SLO) is a measurable reliability target, such as "99.9 percent of requests succeed over 30 days." An SLO is built on a Service Level Indicator (SLI), the actual measurement. The error budget is the small fraction of allowed failure; when it runs low, you slow risky releases and prioritize reliability work.

Prerequisites

• Service metrics available in Prometheus.
• Agreement on what counts as a successful request.
• Basic PromQL.

Steps

1. Choose service level indicators

Good SLIs are user-facing: availability (fraction of non-error requests) and latency (fraction of requests faster than a threshold). Avoid SLIs the user never feels, like CPU usage.

2. Set SLO targets

Pick a target that matches user expectations and is achievable. For example: 99.9 percent availability and 99 percent of requests under 300 ms, both measured over a rolling 30-day window. More nines cost more; do not over-target.

3. Calculate the error budget

The error budget is 100% - SLO. For 99.9 percent availability over 30 days, the budget is 0.1 percent. In time terms that is about 43 minutes of full downtime per 30 days, or the equivalent in failed requests.

4. Express SLIs in PromQL

Availability SLI:

sum(rate(http_request_duration_seconds_count{status!~"5.."}[30d]))
  /
sum(rate(http_request_duration_seconds_count[30d]))

Latency SLI (fraction under 300 ms):

sum(rate(http_request_duration_seconds_bucket{le="0.3"}[30d]))
  /
sum(rate(http_request_duration_seconds_count[30d]))

5. Track burn rate

Burn rate is how fast you consume the budget relative to the SLO window. A burn rate of 1 exhausts the budget exactly at the window's end; a burn rate of 14.4 over one hour would exhaust a 30-day budget in about two days. Alert on a fast burn (high rate over a short window) and a slow burn (lower rate over a longer window) to catch both outages and steady degradation.

Verification

• Each SLI query returns a value between 0 and 1.
• The error budget number is documented and agreed.
• A burn-rate query reacts when you inject errors.

Next Steps

Render SLO and remaining-budget panels in Grafana, add multi-window multi-burn-rate alerts, and adopt a policy: when the budget is exhausted, freeze feature releases until reliability recovers.