The agent patch passed the gates I ran on it. Its unit tests were green, fixtures matched, nothing was flaky. Then I seeded four faults into the implementation, one at a time. Two survived.
That gap is what this article is about. A green suite is a claim, not a measurement. Mutation testing turns it into a measurement: introduce a fault, run the suite, and see whether the suite notices. I now run this loop before merging any agent-written patch, and the whole thing costs a few rebuilds.
Why green tests lie
A passing test proves one thing only: the test and the implementation agree on the inputs the test exercised. When an agent writes both the patch and the tests, the tests inherit the patch's assumptions. If the implementation encodes a wrong assumption, the test encodes the same one. The suite is green because it is blind, not because the code is right.
The patch in this article came from a free model on MonkeyCode's free model access. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The model wrote a bounded queue and a test file. The test file was not wrong. It was blind in exactly the place the implementation was wrong.
