Every team chat I'm in has landed on the same uncomfortable question: an agent can write a plausible patch in ninety seconds, but who vouches for it? Reading the diff by eye feels like due diligence. In practice, it mostly rewards fluency — and generated code is fluent by construction.
The stance I settled on: treat agent output like a pull request from a stranger with no commit history. You wouldn't merge that because it read nicely. You'd merge it because it survived your gauntlet. Below is the gauntlet I run, built entirely on compute that costs nothing, followed by an honest list of where it falls apart.
Why eyeballing diffs keeps letting bad patches through
When I audit agent patches that got a thumbs-up and later caused trouble, three defects recur:
Prompt-shaped correctness. The code nails the scenario in the request and crumbles on the adjacent ones — empty input, DST boundaries, a record with ten thousand rows.
