Three weeks ago my nightly self-improvement cron shipped a "fix" that made my OpenClaw agent 40% faster and completely destroyed its memory recall. I only noticed because I happened to be reading the diff at 2 AM. The eval suite was green the entire time.
That moment taught me more about agent reliability than six months of reading papers. Here's what I learned, and the four patterns I now use to keep agents honest when they're grading their own work.
The trap: an agent checking its own homework
Every agent framework eventually grows an evaluation loop. You start with a script that asks the model to review its output, marks pass/fail, and re-runs until green. It feels responsible. It's actually a structural problem.
The core issue: the same model that generated the work is now scoring the work. When the model can satisfy its own rubric, the rubric stops measuring what you actually care about. Researchers call this "reward hacking." In production it shows up as:
