Field Guide
AI agents: orchestration platforms
09
Common mistakes
1 min read · Chapter 9 of 10
What usually goes wrong
The bill spikes at month end
- Likely cause
- A loop with no turn or budget cap
- Reversal path
- Caps, loop detection; ask for credits
- Who absorbs it
- Customer on consumption; vendor on outcome pricing
The same refund goes out twice
- Likely cause
- Retry or resume without an idempotency key
- Reversal path
- Idempotency keys; compensate
- Who absorbs it
- The deployer
The agent tells a customer a made-up policy
- Likely cause
- Ungrounded answer, no review
- Reversal path
- Grounding checks; review for high stakes
- Who absorbs it
- The deployer
A user sees a document they shouldn't
- Likely cause
- Over-broad permissions; index synced before a revocation
- Reversal path
- Query-time permission checks
- Who absorbs it
- The deployer
Data reaches an attacker
- Likely cause
- Injected content plus a write tool or open network
- Reversal path
- Allow-listed egress; approvals
- Who absorbs it
- The deployer
Production data deleted
- Likely cause
- Broad credentials; no development/production split
- Reversal path
- Least privilege; approve destructive actions
- Who absorbs it
- The deployer and its users
Quality drops after a "minor" change
- Likely cause
- An alias moved, a cloud auto-upgraded, a fallback answered
- Reversal path
- Pin snapshots; regression evals
- Who absorbs it
- The deployer
Calls fail on a date
- Likely cause
- A model or API retired
- Reversal path
- Track dates; re-certify early
- Who absorbs it
- Customer and platform
The agent "remembers" something false
- Likely cause
- Memory written from untrusted text
- Reversal path
- Memory-write rules; rollback
- Who absorbs it
- The deployer and its users
Ranking these by frequency or cost needs an operator's data; I found none public.