Benchmark screenshots are abundant. The narrower question a working C++ team needs answered is harsher: can a coding assistant repair the sort of defect that actually appears in our tree, under our warning policy, without smearing edits across files that were off limits? I stopped asking for impressions and built a tiny grader that turns each model answer into the same three artifacts: a strict build, a hidden behavior check, and a patch-boundary check.
The example below is C++, but the method travels. The model is interchangeable; the gates are not.
Grade the thing a maintainer would grade
A polished demo can hide the exact places where C++ gets unpleasant: warnings that become errors only in CI, bounds that look fine until UBSan runs, header hygiene that differs between GCC and Clang, and fixes that are technically correct yet rewrite half the file. So the unit of evaluation is not plausibility. It is a pass or fail answer to three mechanical questions:
Build gate: does the candidate compile with the same flags your project already uses, including warnings-as-errors and sanitizers?
