The Checkmark Was Never Proof
I don't write test suites for a living. My job is grading them, and grading the AI models that write and judge code in the first place. The rubric I work under does not allow ties. Every verdict needs a named file, a line number, a concrete before and after. "This one feels cleaner" is not a reason, it is a rejection.
Sitting in that seat long enough teaches you something the rest of the industry mostly avoids saying out loud: the green checkmark was never proof of anything. It was always a claim, and I have watched that claim fail the same way in three completely different settings.
Failure one: the retry that hides the bug
Flaky tests are the most visible version of this. Google's own engineering team reported in 2016 that around 16 percent of their tests displayed some flaky behavior, and about 1.5 percent of all test runs across their corpus came back flaky on a given day, meaning roughly one in seven tests written by their engineers would occasionally fail for reasons unrelated to any real code change.
