DiffusionGemma Is Fast Because It Stops Pretending Text Has to Be Written Left to Right
Google DeepMind published DiffusionGemma this week, an open-weight language model that generates text with discrete diffusion instead of the usual token-by-token loop.
That sounds like a paper detail until you look at the serving numbers. The report says DiffusionGemma averages about 20 tokens per forward pass and roughly 1,500 output tokens per second on a single H100. The comparable Gemma 4 autoregressive setup with multi-token prediction lands around 303 tokens per second in the same table.
That number is worth paying attention to. Not because every model is about to become a diffusion model. Because the boring bottleneck in LLM serving is still the bottleneck, and this is one of the cleaner attacks on it.
Autoregressive models are easy to reason about. The model writes the next token, then the next token, then the next token. That left-to-right contract is also the tax. Even with speculative decoding, the target model still has to verify a draft sequence, and the useful speedup depends on how much of that draft it accepts.
