Reliability & Safetyintermediate13 min · 9 steps
Reliability Patterns
Retry the right errors, stop hammering a dead provider, isolate workloads, set layered timeouts and degrade gracefully when everything fails.
requesterrorsignalresponseheavy work
ProblemStep 1 of 9
LLM APIs fail loudly, and sometimes silently
Rate limits, spend caps, timeouts, outages, retired model IDs, context overflow and malformed output all throw. Two failures don't: a quality change under the same model ID, and a safety refusal that arrives as HTTP 200 with empty content.
Set an availability and latency target per tier, then design as if the provider will go down, because it will.
| Tier | Availability | p99 latency |
|---|---|---|
| Critical (payments) | 99.99% | < 3 s |
| Standard (support) | 99.9% | < 10 s |
| Best effort (background) | 99% | < 30 s |
What changed
- +Failure modes, loud and silent
Lesson 3 of 4 in Reliability & Safety