A green test run is not a behavior spec. An agent patch can keep every existing assertion passing and still change encodings, error types, empty-input handling, or the bytes written to stdout. Shadow-compare the candidate against a frozen baseline on the same corpus. Merge only after every divergence is classified in an accepted-delta ledger.
This article is a testing workflow, not a model bake-off. The harness below is labeled as a proposed, runnable pattern. It does not claim production timings, model names, or pass rates.
Why green CI misses the patch
Agent patches optimize for the tests they can see. Hidden behavior lives in branches the suite never names: trailing newlines, NaN keys, timezone-naive stamps, None versus []. Those are cheap to alter. They are expensive to notice after merge.
A dual-run gate treats the old artifact as the oracle for unspecified behavior. Specified behavior still belongs in ordinary tests. The ledger exists for the remainder: diffs you accept on purpose, and diffs you refuse.
