Originally published at kunalganglani.com — read it there for inline code, hero image, and live links.
597 milliseconds. That's how long Claude Haiku 4.5 takes to deliver its first token on a medium-length prompt. GPT-4.1 Mini? Roughly 2,400ms. Four times slower. That gap is the difference between an app that feels alive and one that feels broken. I tested five LLM APIs for latency — Claude Haiku 4.5, Claude Sonnet 4, GPT-4.1, GPT-4.1 Mini, and Gemini 2.5 Flash — across time-to-first-token (TTFT), output throughput, and end-to-end response time. Here's the real data, updated with mid-2026 context.
Vendor benchmarks are marketing. They're run on optimal infrastructure, with cherry-picked prompts, under conditions no production app will ever see. After shipping production AI features that serve real users, I've learned the hard way that the only numbers worth trusting are the ones you measure yourself. So I did.
LLM API Latency Tested: 5 Models, 3 Prompt Sizes, Real Numbers
The benchmark I ran in March 2026 was deliberately simple. Five models, three prompt sizes (short ~50 tokens, medium ~200 tokens, long ~500 tokens), three runs each, all using streaming APIs to capture accurate TTFT. I disabled Gemini's thinking mode for a fair comparison. Every test ran from the same Toronto-based server, same network, same time window.
