Cloud TPU v6e-1 (ct6e-standard-1t, one v6e chip, 32 GB HBM), GCE flex-start, europe-west4-a. vLLM baseline measured 2026-07-21.
The workload nobody benchmarks
Serving benchmarks optimize for the wrong shape. They report throughput at concurrency 100 with 1,024-token prompts, because that's what a public inference endpoint looks like.
A small agent workload looks nothing like that:
Low concurrency, high value per stream. Two to eight agents, not two hundred. Each one is a person waiting, or a pipeline stage blocking.
