Most developers evaluate a new AI coding model the same way: open a chat, type "write a REST API", nod at the output, and either subscribe or move on. I have done this too, and it is a terrible method. The output always looks competent on the first prompt, and the model's real weaknesses only show up on the fifth refactor, the ambiguous requirement, or the codebase question it answers with confident fiction.
This article is a small, repeatable alternative: a fixed suite of eight prompts, a scoring rubric, and a harness that records everything so you can compare models (or the same model a month later) on evidence instead of vibes. You can run the whole thing on free tiers — I will note one option below — so cost is not an excuse to skip it.
Why one-shot prompts mislead you
A single prompt conflates three different capabilities:
Fluency — producing syntactically plausible code. Nearly every current model passes this.
