Model Routing Is Simple. Until It Isn’t.

Back to Articles

1. Cost Is More Than Model Pricing 2. Complexity Is More Than Task Difficulty 3. Latency Is More Than Model Speed So How Did We Handle This? The Bigger Picture Acknowledgement Building a router into your agent sounds like an easy win. Send simple requests to cheaper models, reserve expensive ones for harder tasks, or route by specialty — Claude for code, Gemini for multimodal, and so on. A classifier or heuristic makes the call, costs go down, performance stays up. Done.

Except it’s not. Most routing systems assume that model selection is a classification problem. In our experience building routing into agentic systems, what looks like a model-selection problem quickly becomes a systems optimization problem. Three dimensions made this surprisingly hard for us.

1. Cost Is More Than Model Pricing

We expected GPT-4.1 to be cheaper than Claude Sonnet 4.6. It wasn’t.