The Gate Rejected Nothing. So I Injected 40 Bugs to See If It Could.
An agent-patch gate that never rejects a patch is a hypothesis, not a control. You cannot measure what it catches until you give it something to catch. So I stopped trusting real patches and started manufacturing failures.
For the last two weeks I have been running a three-stage review pipeline for agent-written code: property checks, pinned fixtures, and a freeze on flaky tests. Every real candidate passed. That was the problem. Passing real candidates only proves the sample was easy. Agent patches are easy by construction — the model sees the test suite, and it writes code that satisfies exactly what the tests say. The failures live in what the tests do not say: boundary values, error handling, fixture drift, nondeterminism.
The method that exposed those gaps is mutation testing applied to the gate itself. Take a clean patch, inject a known defect, run the full gate, and record whether the defect is caught. Repeat across bug classes. The output is a detection matrix that shows blind spots instead of vibes.
Pass rate is the wrong number
