Valentyn Kropov, ​СTO at N-iX, a global technology partner for Pragmatic AI Software Engineering.gettyMy team has been working with a global technology leader in industrial manufacturing to build a shared AI gateway for tens of thousands of engineers using multiple approved AI providers. As adoption scaled, so did a less visible problem—small costs multiply fast once thousands of people are doing it at once. It's a bit like every employee in a company using a taxi instead of the bus. A single ride doesn't break the budget, but doing it repeatedly does.This experience reinforced what we increasingly see across enterprises: AI cost optimization starts with measuring what a business outcome truly costs to produce, then designing the right mix of model routing, context management, caching, batching and platform-level controls around that number. It's also why, in our AI consulting practice, we encourage clients to put the governance layer in place before the costs compound.In this article, I share what AI cost optimization means in practice and what keeping costs under control looks like once you're past the pilot stage.​​You can’t optimize what you didn’t measure.From what I see, most enterprises still can't say what their AI spend returns. It may be the largest unmeasured investment in enterprise IT right now. Measurement comes first—that's one of the core ideas in our own AI adoption framework, APEX.But what should you measure? Tokens are an input. The numbers leadership can act on are the costs to resolve a ticket, screen a transaction and process an invoice. So, measure cost per outcome, not just tokens.Capture the cost per outcome of a workflow before you change anything, so every later measurement is a delta against a known starting point instead of a guess. Sequence your changes by where a delta will show fastest, so the first result lands in weeks and is clean enough to hand to the next team.Think in model portfolios.Early deployments route everything through one capable model. Optimization begins the moment you start differentiating. You wouldn't assign a junior’s task to a senior specialist, and the same applies to models.​The practical move is to measure. Pull a week of real production traffic and label each request by what it demands. In most enterprise workloads, a large share is retrieval, classification, extraction, routing and templated generation, all of which an 8-billion or 30-billion-parameter model handles well once it has good context. The hard work is ambiguous judgment, multistep planning and nontrivial code.When we run that exercise with clients, the share that can safely move to a smaller model is routinely higher than the team assumed, often more than half the workload.​ Then, we wire the routing so each request lands on the cheapest model that still clears its accuracy bar, and we reserve frontier models for the hard cases. Set that bar with an evaluation set, and monitor it in production, so you can spot even the slightest drops in measurable quality before a customer finds it. Prices and capabilities should change every few weeks, so the routing table becomes a living asset, with a clear owner.Invest in context; spend less on reasoning.A lot of the reasoning you pay for is the model compensating for missing context. Give it your proprietary knowledge—structured, permission-aware, current and searchable—and a small model with a tight prompt will beat a large model guessing. Retrieval quality is where we put the engineering effort first because it’s the cheapest lever available.After that, two techniques compound the savings, and neither asks you to rebuild the logic.Caching prevents you from paying twice for the same intelligence. Prompt caching reuses the fixed portion of a long instruction; semantic caching recognizes when a new question is close to one already answered and returns the stored result. In high-repetition areas such as HR and IT help desks, a large fraction of traffic is near-duplicate, and every cache hit is a saving.Batching applies to all nonurgent work. Compliance sweeps, reporting and back-office document processing can tolerate minutes of delay. Grouping them drops the cost per request. Tag each workload by real business urgency, and stop paying real-time prices for work that has no real-time urgency. That tagging is an organizational task, not a simple config change. That's why it needs the teams who own those workloads involved.Solve it once, at the platform layer.The AI gateway is a centralized layer between enterprise applications and AI models—the control plane through which AI requests pass. That's where you route to the right model, replace an LLM call with deterministic logic and apply caching and batching. The gateway serves both companywide AI standardization and economics. This layer handles both cost control and CIO governance.At the beginning of this article, I mentioned that we're working with a global technology leader to build a shared gateway for AI tool servers. The goal is to let their thousands of developers connect to an approved MCP in a single line of code, from any AI client, to optimize cost.​To do this, we stopped handing the model every tool at once. Instead, we expose two operations, search and call, and the model pulls in only the handful of tools a given task needs. We took the idea from Anthropic's Tool Search, moved it up to the gateway and kept it provider-agnostic so it isn't tied to any single vendor. For a company that size, avoiding lock-in is worth as much as the tokens it saves.The cheapest model isn't the only lever. Keeping the context you send small matters as much, and the platform layer is where both get solved once and applied everywhere.None of this is glamorous work, and at enterprise scale, it doesn't need to be. Even a few cents per request multiplied across millions of AI calls is a line item the CFO will notice. Before your next model decision, ask yourself one question: What does an outcome cost today, and who owns that number? If nobody can answer, that's where the work starts.​Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?