You change one line of a prompt and have no idea what you broke, because diff is useless on free text. The fix is to stop comparing text: freeze a set of real traces, assert on behaviour — which tools got called, which constraints held, what the agent refused to do — and run that suite on every prompt or model change.
Concretely, three files and a command. A cases.jsonl of frozen inputs, an assertions.ts that checks behaviour rather than strings, and a runner you invoke after each change. If you work in Claude Code or Cursor, this is a good candidate for a slash command, so the check costs you one line instead of a context switch.
How it works
The unit under test is not the final message. It's the trace: the ordered record of what the agent did to produce that message — tool calls and their arguments, retrieved documents, the number of turns, tokens spent, and the final output. A trace is structured data, and structured data can be asserted on.
Four kinds of assertion cover most real failures:
