Last week I wrote about tracelint, a linter that catches structural bugs in agent traces — the classic one being an agent that calls charge_card, gets a failure back, and just... keeps going and tells the customer their order shipped.
The response was better than I expected, and one question kept coming up in different forms: "Okay — but wouldn't my eval suite have caught that anyway?"
Fair question. So I checked. On the case I'll show you, it didn't. The suite was green. The charge was declined. The agent said "payment successful," and every eval passed.
This post is about the tool I used to find that — muteval — and what happened when I pointed it at the exact failure tracelint was built for.
The blind spot in a passing eval suite
