Every voice agent does the same three things. It listens, it thinks, and it talks back. The interesting part isn't the what — it's the how. And the "how" splits into three very different architectures, each with its own trade-offs on latency, control, cost, and the one thing that quietly decides whether your agent is any good: accuracy.
So before you wire anything together, it's worth understanding the three patterns you can choose from — and why the step everyone treats as plumbing is actually the ceiling on how smart your agent can be.
The job of a voice agent: hear, think, speak
Strip away the framework names and a voice agent is a loop. A person says something. The agent turns that audio into text (speech-to-text, or STT). A language model reads the text and decides what to say (the LLM). That response gets turned back into audio (text-to-speech, or TTS). Then it waits for the next thing the person says and does it all again — ideally in about a second, because anything slower feels like a bad phone connection.
Here's the thing most architecture debates miss: the model that thinks is only ever reacting to what the first step heard. If the STT layer mishears "my account ends in 4-0-1-5" as "4-0-1-8," the LLM confidently helps with the wrong account. Get the input wrong and everything downstream is wrong too. Keep that in mind — it's the theme that runs through all three architectures below.
