Voice AI agents are magical until they start lagging. Suddenly my Groq-powered voice assistant started taking 3 seconds to respond and standard APM tools were useless. They could tell me that an HTTP request was open, but they couldn’t tell me if the bottleneck was the Speech-to-Text (STT) transcription, the LLM’s Time to First Token (TTFT), or the Text-to-Speech (TTS) synthesis.
Here’s how I did this by building Zooid, a custom OpenTelemetry instrumentation layer that traces streaming Voice AI, tracks FinOps token costs, and visualizes everything natively in SigNoz.
The Challenge: Web Tracing or Voice Tracing
A typical CRUD web application is easy to trace. A request comes in, hits a database, and returns a JSON response. You drop in the opentelemetry-instrumentation-flask package, and you are done.
Voice AI is a whole other beast. A "Voice Turn" consists of:
