Field Guide
Voice AI: real-time voice agents
09
Common mistakes
1 min read · Chapter 9 of 10
What usually goes wrong
| Symptom | Likely cause | First thing to check |
|---|---|---|
| Callers say the agent is slow | Long model response time, tool round trips, high reasoning effort, hops across clouds | Mouth-to-ear time at the 95th percentile, by stage |
| The agent talks over callers | Silence timer too short; slow speakers; callers reading digits | End-of-turn settings during data capture; keypad entry |
| The agent stops for no reason | Backchannels, coughs, a TV, its own voice echoing | False-interruption rate; interruption classifier |
| Wrong names, emails or account numbers | 8 kHz audio, accents, noise | Read-back confirmation; spelling and keypad fallbacks |
| Calls end mid-conversation at 10 minutes | Platform maximum duration or a model connection limit | Max-duration settings; session resumption |
| Outbound calls go unanswered | "Spam Likely" labels, weak attestation, number reputation | Registration with analytics engines; your own signing token; SIP 603+ responses |
| A customer got two calls | An ambiguous timeout retried automatically | Dial logic and retry rules |
| The bill grows with call length | Speech-to-speech context replay; cache broken by mid-call changes | Cached share of input; static instructions and tools |
| A lawyer's letter about recording | The vendor can train on audio; no notice naming it | Retention and training settings; the greeting |
| A cloned voice gets through | Voiceprint as the only check | A second factor for money movement and account changes |