A test suite that passes after an agent patch proves one thing: the patch satisfies the tests you wrote. It says nothing about the behavior users already depend on. The cheapest reliable oracle is differential — run the old code and the new code on the same inputs, compare outputs. This article shows a gate built on that idea, plus a change manifest for the diffs you actually want.
Why "the agent's tests pass" is the wrong signal
Agents patch toward the tests in context. That is the failure mode.
Consider the common case: an agent "fixes" a slow lookup by adding a cache. Every existing test passes, because each one calls the lookup once. State is invisible. The patch changes the answer for the second call on the same key, and no test asks about the second call.
The pattern deserves a name: unit tests assert single-call correctness. Agent patches introduce caches, retries, ordering, and shared state. Differential testing catches those changes because it compares call sequences, not single calls.
