Back to Articles
This article is complementary material for Class 3: Reinforcement Learning of the Training an Agent series Ben and I are doing, where we train a coding agent step by step. This article does not explain how GRPO works. The class does that, with the three experiments. Here we show where the same ideas appear in the reports of frontier labs. Previous classes: SFT on traces and distillation.
It also pairs with Distillation in 2026 (so far), where we did the same exercise for distillation.
Something I really like when I study a subject is understanding its history, how it reached the point where it is today. Having at least a glimpse of where things come from adds a lot to my own understanding. This piece tries to do that for RL in post-training, and to keep it accessible. The goal is not to be exhaustive. It is to situate ourselves and set a common ground.
In the live session, Ben explained in depth how GRPO works, and then we went through three experiments: one with a toy reward, one with a verifiable reward, and one with a reward we broke on purpose. In that last run, the Trackio dashboard said the training was going great while the model got worse at coding. We tell the full story of that run in the recording.
