The headlines say enterprises can’t control their inference spend — the sometimes-surprising amount they pay frontier model companies to use their LLMs. Uber burned through its entire AI budget in four months. GitHub’s usage-based pricing drew an instant backlash when customers realized how much more they would have to spend.
Now, companies are cutting inference costs by routing to cheaper rented models, and engineers are optimizing their existing models to reduce the bill. But cost is just the most visible symptom of a much larger problem. When a single rented model runs every job for your agent, from intent detection to safety to citation to policy, it’s a chokepoint for your whole business, not just your invoices. The latency sits at the model provider’s mercy, the roadmap depends on their release schedule, and the safety logic lives in a black box. Obsessing over token spend is a local minimum. The real engineering problem is architectural agency.
We know that pressure firsthand. Eighteen months ago, Agentforce also ran entirely on a single rented model, and our token bill grew linearly with traffic. We could have passed the rising bill on to our customers, but that would have bought us a year at most. The bill would keep rising as long as we were renting the whole engine. So instead of repricing, we opted to rebuild. Instead of handing everything to one model, we broke out different tasks and tuned specific open-source models to accomplish them.
