1. The Starting Line: The Last Non-Reasoning Flagships
GPT-4.5, DeepSeek-V3, and Claude 3.5 Sonnet share something that has nothing to do with benchmark scores: they were the last major models built entirely on the "pretrain, then instruct-tune" recipe. All internal computation was done in one forward pass per token, with no backtracking, verification or revision mechanisms in place.
By late 2024, these labs had run into the same big wall: scaling pretraining data and compute had reached a point of saturation. The naive recipe of more tokens, bigger model, more GPU-compute no longer bought equivalent capability gains, particularly on multi-step reasoning tasks. Good output quality became the bottleneck before parameter count did.
What came after wasn't a bigger version of the same thing. It was a different training paradigm applied on top of the existing ones. The field pivoted to training models to think before they answer.
2. RL Terminologies Recap
