I did not get to this opinion from a whitepaper.
I got to it because I am building an open-source project around the problem, and the build keeps arguing back.
I am working on AgentEval Forge right now, an OSS evaluation lab for agents. The original idea sounded straightforward enough: scenario packs, adversarial cases, trajectory scoring, regression tracking, cost and latency analysis. In my head, it was going to be a more serious, more structured version of evaluation work I had already done elsewhere.
That was the first mistake.
I had already spent time building model-eval and workload-eval style systems. I knew what it meant to compare models for real tasks, how to design rubrics, how to think about speed versus cost, and how quickly a neat score can become misleading if the harness is weak. I assumed agent evaluation would be an extension of that same world.
