We measured a week of inference. Routing by task difficulty cuts our cost per call roughly 48x — and flips which users are profitable.

We did the thing everyone building on LLMs does. We defaulted to a strong frontier model, because the demo has to be good and nobody gets fired for picking the strongest model. Then we measured a week of production traffic, and the numbers were embarrassing enough to write down.

One frontier model was carrying 77 percent of our calls and 97 percent of our inference spend. Not because 77 percent of the work needed it. Because it was the default, and defaults are where money goes to hide.

This post is the arithmetic that fell out of that measurement, because I think the shape of it generalizes to almost anyone running an AI product, and because the interesting part is not the cost reduction. The interesting part is what it does to your margin curve.

Start with the shape of your traffic, not the price sheet

Provider price pages quote dollars per million tokens, which is a unit nobody's intuition works in. The number that matters is cost per call at your measured shape, and you cannot get it from the price sheet alone — you have to instrument production and look.