Last week, a Baseten inference engineer who goes by @waterloo_intern published a technical blog post titled "22,580: From GPT-2 to Kimi K3, Explained." It hit 2.4 million views in days.
He didn't write a press release. He wrote runnable PyTorch code — starting from GPT-2's attention block, stepping through every architectural change, explaining one problem and one cost per iteration. It's the best transformer lineage explanation I've seen.
I devoured his post, then cross-checked the key claims against 5 original papers. Here's the full picture.
The 22,580x Number
In February 2019, OpenAI released GPT-2 — 124M parameters. Seven years later, Moonshot AI open-sourced Kimi K3 — 2.8T parameters. You could fit 22,580 GPT-2s inside one Kimi K3.
