Building a new model from scratch wasn't necessary. The team started with the existing Gemma-4-26B-A4B and converted it into a diffusion model using less than ten percent of the original training token budget, according to the report.
DiffusionGemma delivers several times the output speed of the Gemma 4 models and previous diffusion models while maintaining comparable accuracy. | Image: Google
Two training stages balance quality and speed
In the first of two steps, the model learns to reconstruct noisy text blocks from example data. A combined phase of reinforcement learning and sampler distillation follows, which Google calls SD·RL. Reinforcement learning typically boosts answer quality, while sampler distillation lets the model get by with fewer compute steps. Google merges both into a single process.
Google DeepMind doesn't train DiffusionGemma from scratch but converts the finished Gemma 4 model through two training stages. | Image: Google
