Tracking LLM Latency & Cost: How I Instrumented an AI Agent Pipeline Using OpenTelemetry and SigNoz

The Problem: The High Cost of Blind AI Requests

When you build a standard CRUD application, performance bottlenecks are predictable: it's almost always a missing database index or a heavy payload. With AI agents and LLMs, performance is wildly unpredictable.

A single user prompt can trigger multiple API calls, prompt formatting, vector database lookups (RAG), and streaming responses. If a user complains that the app is slow, you can't just check your server CPU usage. You need to know:

Which specific LLM call took the longest?

Did our vector database search slow down the context retrieval?