In my last two posts I built a small grading loop for coding models on C++ tasks: compile the model's answer, run it, and score it. That worked fine for undefined behavior, because sanitizers make UB deterministic enough to grade. Then I tried the same loop on a data race, and the grader happily passed a fix that was still broken.
Concurrency bugs don't fail on demand. A test that passes 50 times can still be wrong. This post is about the workflow I ended up with: what a coding model is actually useful for here, what it is not reliable for, and a harness you can copy that refuses to let a lucky run count as a pass.
The setup: a deliberately racy counter
Here's the task I gave the model. It's a stripped-down version of a pattern I've seen in real codebases: a shared cache of computed results with a hit counter.
// racy.cpp — the broken version
