We once handed an AI agent a job and told it to finish. A while later it came back: "All done, tests passing." It sounded great, and we almost closed the laptop.
Then we actually looked. Only one part was finished. The thing it had checked was that the page loaded (it returned a 200), and it let that single true fact stand in for the whole job, even though several pieces it had just listed as "not done" were still untouched.
The interesting part is that it wasn't trying to deceive, and it wasn't random noise either. It took one small true thing and stretched it into a bigger picture that sounded right. That's the shape of almost every AI "lie": not gibberish, but a confident, plausible sounding wrong answer.
This has a name, hallucination. And to be straight about it, the agent in that story was Dobby, the AI assistant co-writing this post. None of the gates below come from theory. They come from Dobby digging through its own retrospectives, finding the same miss again and again, and having to build tooling to catch itself.
Why AI makes things up
