I Ran 10 AI Coding Models Through 5 Tasks: A Data Scientist's Take
I'll be honest — I went into this expecting a clear winner. I came out with a scatter plot, three regressions, and a deeper appreciation for why "best" is the most dangerous word in machine learning.
Over the past three weeks I've been grinding through prompts with ten different LLMs, all routed through the same endpoint, scoring every output on a 1–10 rubric that I tried very hard not to bias. The pricing data is pulled directly from the provider pages. The scores are mine. If you disagree with a score, you're probably right — n=1 per task per model is a laughably small sample size, and I say that as someone who publishes papers with bigger samples. But trends still emerged. Let me walk you through what I found.
The Lineup
Before I touch a single benchmark, here's the cast. I've grouped them by family so you can see the obvious concentration in the open-source Chinese ecosystem, which personally I find fascinating — three of the top five are DeepSeek or Qwen variants.
