For five years, the answer to "how do we make the model better" was always the same: bigger model, more data, wait for the loss curve to bend. Scaling laws made AI progress feel almost boring — predictable, like compound interest.
That knob still works. But it's not the interesting one anymore.
The real question teams are fighting with now is: given a model you already have, how much compute should it spend per question? Not per training run. Per question. And the answer turns out to break almost every assumption your inference stack was built on.
The single forward pass was always the weak link
Here's the thing nobody says out loud enough: a model's first answer is often just a noisy sample. The right answer was in there somewhere — it just wasn't the loudest one. You don't fix that with more pretraining tokens. You fix it by sampling more, checking the work, or letting the model take a second pass.
