Every AI gateway adds a hop between your app and the model. The question that matters is what that hop costs at the moment your user is staring at a blank chat window: the time to first token. Most gateway latency debates skip the measurement and argue architecture — so we measured it.
We ran an open-source TTFT benchmark against LLM Gateway and OpenRouter, interleaved, from the same machine, on the same model. The medians over 75 runs each: LLM Gateway got the first content token in 906ms on a cold connection and 814ms on a warm one. OpenRouter took 1392ms and 1232ms. That is roughly 35% faster cold and 34% faster warm, with zero errors across the 300 measured runs — 450 HTTP requests in total, counting the throwaway warm-up call that precedes each warm measurement, every one of which returned HTTP 200. The raw per-run data is published in full.
How we measured AI gateway performance
We used ai-gateways-benchmark, an open-source script by Ronny Badilla that recently made the rounds comparing Vercel AI Gateway, OpenRouter, and Cloudflare AI Gateway. It is Python stdlib only — raw sockets, no HTTP library — and it times every phase of a streaming request separately: DNS, TCP connect, TLS handshake, TTFB (request sent to first response byte), and TTFT (request sent to first content token in the SSE stream).
