I can make almost any agent change look good with one carefully chosen prompt. Run it once. Watch...
I can make almost any agent change look good with one carefully chosen prompt.
Run it once. Watch the right tool fire. Read a polished answer. Record the demo.
Ship it?
Not yet.
A single trace is excellent evidence for one execution: what ran, what failed, how long it took, and whether declared invariants held. A release decision asks a different question:
Agent Runs Are Black Boxes: A Trace-First Debug Loop As agents take on more coding work,...
A green test suite is not real evidence. It is often a closed argument loop. The same agent wrote...
You shipped your agent. Evals were green. A week later you tweak the system prompt to fix one...
A few months ago I wrote about regression-testing prompts before shipping them. The responses that...
When a single agent does something dangerous, the audit problem is small. You have one run, one set...
Most AI agent demos stop at the final answer. Mine did too at first. I had a support agent that...
