A developer recently asked a coding agent to write a test suite for a function that parses ISO dates. The agent returned fourteen test cases, every one green, and the developer merged them without a second look. A week later, a colleague extended the function to accept two additional timestamp formats, and the suite still passed even though both new formats were broken. The tests had been asserting the same happy path with slightly different inputs, and none of them pinned down the edge cases the function existed to handle.
The current conversation around AI coding agents focuses on the code they generate, but the tests they write deserve the same scrutiny. A green suite from an agent is not evidence of a meaningful suite, and the distinction matters because false confidence is more dangerous than no tests at all. Mutation testing offers a rigorous way to measure the difference, and a free server with a generous token allowance makes the measurement cheap enough to run on every agent-generated suite.
Mutation testing works by introducing small, deliberate bugs into the source code and checking whether the test suite catches them. Each deliberate bug is a mutant, and the mutation score is the percentage of mutants the suite kills. A suite that scores below fifty percent is mostly decorative, no matter how many assertions it contains. The technique is well established, but it has a reputation for being slow, which is exactly where a disposable remote workspace changes the economics.
