Field Guide
Models and inference: what AI runs on
09
Common mistakes
1 min read · Chapter 9 of 10
What usually goes wrong
| Symptom | Likely cause | First thing to check |
|---|---|---|
| Bursts of 429 errors | Rate limits per model, retries without jitter, a fast ramp | Rate-limit headers, retry logic, a second provider |
| The feature stops until the 1st | Monthly spend cap reached | Tier caps and alerts at 50% and 80% |
| The bill jumps with no traffic change | Reasoning effort raised, cache broken by a prompt edit, a new tokenizer | Tokens per request by type; cache hit rate |
| Answers got worse with no release | A serving change at the provider, or a failover to another host | Online evals by model and host |
| A model is going away | Retirement notice | The regression set; pinned snapshot IDs |
| Legal blocks a route | Retention, residency or indemnity doesn't cover that model | The data and indemnity terms per model and venue |
| A gateway leaks keys | Supply-chain compromise | Pinned package versions; key rotation |
| Self-hosting costs more than planned | Low utilization, idle redundancy, people | Tokens per GPU-hour against the provider's price |