Home
Storia in 1 fonti
g5g vs g6 for LLM Serving: the Same Code, and 3.7x the Throughput
Serving Gemma 4 E2B in pure JAX on AWS g5g.2xlarge and g6.2xlarge with a byte-identical payload. The older instance loses 87% of decode to dtype conversion, and nothing in the logs says so.
Raccontata da
dev.to
Timeline cronologica
- ·
Pure JAX on G5g: Serving Gemma 4 on Graviton and a T4G
This article provides a step by step deployment guide for serving Google's Gemma 4 on an AWS EC2 G5g...
- ·
Gemma 4 in Pure JAX: What Changes Between Turing and Ada, and What Doesn't
One hand-written Gemma 4 port, no PyTorch and no vLLM, on two NVIDIA GPUs a generation apart. Most of it ports untouched. Two things do not, and one of them was quietly eating 87%…
- ·
g5g vs g6 for LLM Serving: the Same Code, and 3.7x the Throughput
Serving Gemma 4 E2B in pure JAX on AWS g5g.2xlarge and g6.2xlarge with a byte-identical payload. The older instance loses 87% of decode to dtype conversion, and nothing in the…