Every other week it feels like a new model shows up with a shiny score on some "trust me bro" benchmark. The numbers climb, people call it smarter, and suddenly you're ready to switch. If you build applications on top of these models, you might catch yourself assuming that higher benchmark score means better agent performance. But benchmarks measure narrow skills under controlled conditions; your agent has a specific job to do. If you're building a customer support agent, you don't care about a few extra points on a reasoning test. You care about whether it calls the right tool, passes the right arguments, and avoids risky or redundant actions. That's the behavior that actually matters.
Testing a tool-calling agent means testing its decisions, not just its words. An agent can sound perfectly convincing while quietly messing things up underneath. When it answers "Your refund has been processed", the response sounds fine but did it check the right order? Refund the correct amount? Accidentally hit the refund API twice? The final response won't tell you.
What you actually need is an eval suite: a set of automated tests that score how well an agent performs. Eval is short for evaluation: each test runs the agent on a scenario, asks a grader "was this behavior correct?", and turns the answer into a score you can track and gate on. Traditional software tests and prompt-and-response evals both fall short here. A unit test asserts that a function returns the right value for a given input. An agent, though, makes a sequence of decisions (which tool to call, with what arguments, in what order), and each decision is non-deterministic. The same prompt can produce a different tool-calling path on every run. A single manual test tells you what the agent can do, not what it typically does.
