A green CI badge measures one thing only: the tests you already wrote still pass. It does not measure whether an AI-generated patch preserved the behaviors you never wrote down. When the model read those tests as part of its prompt context, the badge is scoring a target the patch was already aiming at. My position is direct: for free-model patches, the repo test suite is a compromised oracle, and the only gate that survives is a contract probe the model never saw.
Generation stopped being the bottleneck the day free model access arrived. A patch now costs tokens instead of budget, and the free server option gives you a disposable runtime where that patch can execute before any human reads a line. That shift moves the entire risk to verification, which is exactly where most teams are least prepared. They reach for the test suite that already lives in the repository, and that is the mistake.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
The suite is compromised in a mechanical sense, not a moral one. When you hand a model your repository, the tests become part of the context it optimizes against, so a patch that satisfies them is pattern matching, not evidence. The suite also encodes yesterday's assumptions about the system, and a model that reads those assumptions can reproduce them faithfully. Meanwhile the invariants no test ever captured — idempotency, isolation, timeout behavior — break silently behind a green badge.
