One hand-written Gemma 4 port, no PyTorch and no vLLM, on two NVIDIA GPUs a generation apart. Most of it ports untouched. Two things do not, and one of them was quietly eating 87% of decode.
This article is a measurement report on running a hand-written Gemma 4 port in
pure JAX across two NVIDIA GPUs a generation apart, and on the two places the
"it's just JAX" abstraction leaks. One of those leaks costs 87% of decode and
nothing in the logs is red.
The code is here:
This article is about running a hand-written Gemma 4 port in pure JAX on three different...
This article provides a step by step deployment guide for serving Google's Gemma 4 on an AWS EC2 G5g...
Serving Gemma 4 E2B in pure JAX on AWS g5g.2xlarge and g6.2xlarge with a byte-identical payload. The older instance loses 87% of…
A field report on running Google's Gemma-4 on AWS Inferentia2: mixed attention heads, the vLLM / optimum-neuron / NxD dead-ends,…
A Blog post by Nikhil K. on Hugging Face
I pulled the just-released Gemma 4 12B and ran it on a GTX 1080 Ti. ~28 tok/s at Q4 on one card — but three things broke first,…
