A few weeks ago I wrote about building a reproducible test harness for comparing free AI coding models before you commit. That harness answers one question: which model should I use?
It does not answer the harder follow-up: once a model generates a patch for my real codebase, when is it safe to merge?
This week there was a great discussion on DEV about "understanding over origin" — the idea that it doesn't matter whether code came from a human or a model, only whether someone actually understands it. I agree with the principle, but principles don't survive contact with a busy afternoon. What survives is a checklist with teeth. So here is the pipeline I bolted onto my model harness: every AI-generated patch has to pass through a scripted review gate before I even read it, and the script produces a scorecard that tells me how carefully I need to read it.
The problem with eyeballing diffs
When a model produces a 40-line diff that looks idiomatic, my brain does a dangerous thing: it pattern-matches on style and skips semantics. The code reads like something I'd write, so I approve it like something I'd write. The failures I've actually shipped from AI-generated code were never syntax errors — the tests even passed. They were things like:
