A field report on serving Gemma 4 E2B under vLLM on AWS G5g — the only aarch64 + SM 7.5 hardware there is. No published build covers that combination, AWS quietly solves half of it, and the thing that actually blocks you is 64 KiB of shared memory.
A field report on serving Google's Gemma 4 E2B on AWS EC2 **G5g* — a Graviton2 (aarch64)
host with an NVIDIA T4G (Turing, SM 7.5) GPU. Three obstacles: an arch list nobody
publishes for this combination, a version floor that only the newest vLLM clears, and
64 KiB of shared memory that stops the model dead. Plus the seven things I documented
wrong before I had a box.*
A field report on running Google's Gemma-4 on AWS Inferentia2: mixed attention heads, the vLLM / optimum-neuron / NxD dead-ends,…
This is a submission for the Gemma 4 Challenge: Write About Gemma 4 TL;DR: You don't need an RTX...
This is a submission for the Gemma 4 Challenge: Write About Gemma 4 Google released four Gemma 4...
How to get Google's Gemma 4 26B-A4B Mixture-of-Experts model running locally — including speculative...
Step-by-step: getting vLLM's Rust frontend built and running on an aarch64 EC2 G5g box. rustup, setuptools-rust, protoc, the…
Learn how to deploy Gemma 4 26B on legacy CPU hardware using quantization and optimization techniques for surprisingly effective…
