Every team I talk to says their agent "sometimes hallucinates," and almost none of them can tell me how often. That gap — between knowing it happens and being able to count it — is the whole problem. You cannot fix, gate, or even trend a failure mode you only detect by feel.
Here is the opinion I will defend: hallucination detection is not a model-quality problem, it's an instrumentation problem. The reason you can't measure it is that you threw away the evidence the moment the agent finished running. Detecting an ungrounded claim requires knowing what the agent was allowed to claim, and that lives in the tool outputs and retrieved context, not in the final answer string. If you don't capture those, every hallucination check you write is guessing.
Let me break down what hallucination actually is in an agentic system, why the popular detection methods miss the common case, and how to wire up a number you can put in CI.
"Hallucination" is three different bugs wearing one coat
The word is overloaded, and the overloading is why detection efforts flail. In a tool-using agent, there are at least three distinct failures people lump together:
