Do Not Add an LLM Router Until You Can Defend Each Downgrade

Cost optimization is not a model-picker problem

When an LLM bill grows, the tempting answer is a blanket model downgrade. That is usually the wrong first move. A support classification, a retrieval rewrite, and a complex planning step do not have the same failure cost.

The useful question is: which calls have evidence that they can be cheaper?

Frugon is an MIT-licensed, Python 3.10+ CLI that analyzes OpenAI-compatible request/response JSONL logs. Its stated purpose is to compare candidate models, estimate costs, and propose a split between calls that might move and calls that should remain on the baseline. See the README, project configuration, and v0.2.4 release.

The useful design choice: analysis before automation

Do Not Add an LLM Router Until You Can Defend Each Downgrade

Related reading

Cutting our LLM bill ~80% with model routing: the actual cost math

LLM router architecture: best practices for 2026

How Semantic Routing Cut My LLM Costs by 70% Without Touching Model Quality

Routing Down Is Easy. Knowing When Not To Is Hard: Why Cheap Models Break Your…

How I Cut My LLM API Bill by 80% With a Simple Router

Cut your LLM bill by 30 to 70%: the levers that work