A production AI voice agent should start speaking within 800ms-1.2 seconds of the caller finishing a sentence. Past 2 seconds, callers assume the line dropped, start repeating themselves, or hang up. If you're evaluating vendors or building this in-house, latency isn't a nice-to-have metric — it's the difference between an agent people trust and one they route around.
Why latency decides whether people trust the agent
Human conversation has a rhythm. Research on turn-taking across languages found the average gap between one person finishing and the next starting is around 200 milliseconds — often even before the first speaker finishes (Stivers et al., PNAS). Nobody expects an AI to hit 200ms today, but the baseline it's competing against is that fast, not the 3-5 seconds a customer might tolerate from a slow web page.
On the interface side, Nielsen Norman Group's response-time thresholds are the classic reference: under 0.1s feels instant, up to 1s keeps the user's flow of thought uninterrupted, and past 10s they've mentally checked out. Voice compresses that scale — because there's no spinner, no progress bar, just silence — so a 2-second gap on a call feels much longer than the same gap on a webpage.
