The annoying thing about voice agents is that “the model is fast” does not mean the experience is fast.

I had a small voice assistant running on a local device, talking to a hosted chat backend. The actual LLM call was only one part of the wait. The full path looked more like this:

wake word detection

speech recognition

authenticated /chat call