Ask an agent how sure it is and it will happily tell you. "Confidence: 0.92." It looks like a probability. It renders nicely in a dashboard. Teams wire it into routing logic: high confidence, auto-approve; low confidence, send to a human. It feels rigorous.
It is not rigorous. A self-reported confidence score is the agent grading its own homework, and it is one of the most seductive false signals in production agentic systems. If you are gating anything on it, you are trusting the defendant's opinion of their own alibi.
Where the number actually comes from
When an LLM emits confidence: 0.92, that number is not a calibrated posterior. It is another token sequence, generated by the same forward pass that produced the answer you are unsure about. It shares a substrate with the output it is describing. If the model hallucinated a file path, the same weights that invented the path will cheerfully assign it 0.9 confidence, because from the inside, a confident fabrication and a confident fact are indistinguishable.
This is not a prompt-engineering problem you can fix with "be honest about your uncertainty." You can push the distribution around, but you cannot make a model's self-report into independent evidence, because there is no independent ground truth in the loop. The grader and the graded are the same network.
