A side-by-side of all five Gemma-4 variants on Inferentia2 — PLE, KV-sharing, MatFormer, mixed attention, and a 128-expert MoE — the recipe that evolved to carry all of them, and the one bug that shows up in every single one.
I ported the whole Gemma-4 family — E2B, E4B, 12B, 31B, and the 26B-A4B MoE — to run on AWS
Inferentia2. Each has its own write-up in this series; this is the map. What's shared, what's different,
how the recipe evolved from "trace it and pray" to a single-rank-compile pipeline that carries a
30-billion-parameter dense model and a 128-expert MoE — and the one bug that appears, in some costume, in
**every* port.*
A field report on running Google's Gemma-4 on AWS Inferentia2: mixed attention heads, the vLLM / optimum-neuron / NxD dead-ends,…
The MoE was the hard one: a dual-path FFN, a sparse expert loop that won't trace, and a bug where the device output was empty…
Porting Gemma-4 31B (dense) to AWS Inferentia2: the tensor-parallel recipe that worked at 12B collapses at 31B, NxD ModelBuilder…
The 12B ships as a multimodal class with no encoder loaded, and its sliding-window attention overflows Neuron's fused-attention…
This is a submission for the Gemma 4 Challenge: Write About Gemma 4 Google released four Gemma 4...
In the last entry I got Gemma-4's 128-expert MoE running on an inf2.24xlarge and signed off with...
