An agent patch can pass every fixture in your repository and still be wrong. The fixtures are part of the oracle, and fixtures can be lazy. If the expected value in a fixture never changes, the suite can approve code that breaks exactly the case the fixture forgot.
This is a three-phase probe for evaluating agent-generated patches. Phase 1 runs property checks before anything else. Phase 2 mutates fixtures to prove they can fail. Phase 3 freezes flaky tests only after the first two phases pass. Order matters.
The probe fits on MonkeyCode's free server option, and MonkeyCode's free model access is useful for drafting the invariant skeleton from a patch's diff. Disclosure: This article was prepared as part of MonkeyCode's product outreach.
Why Golden Fixtures Are Weak Oracles
Golden fixtures assert one output for one input. They catch regressions on paths you already knew about. Agent patches are good at preserving known paths and bad at inventing new cases. If the fixture never expected an edge case, the patched code will happily return whatever it wants.
