RELIABILITY

Reliability Playbook for Production Teams

Operational defaults that keep latency predictable and spend under control at scale.

1. Instrument every call with request IDs and model tags for traceability.
2. Define SLOs for P95 latency and error budgets per product surface.
3. Run canary traffic on new routes before shifting production mix.
4. Maintain circuit breakers with bounded retries and jittered backoff.
5. Review weekly cost variance against token throughput anomalies.