At Datadog, we want to expose our engineers to high-quality AI tools and workflows. However, token usage can be expensive, and finding a balance between AI cloud spend and the return on investment can be difficult. But what if engineers could maintain their current AI workflows using the same tools, but at a lower cost? Similar to rightsizing cloud infrastructure, tuning the configurations of AI tools—such as model type, modal families, and effort level—can yield cost savings without hurting performance objectives.
In this blog post, we’ll discuss a handful of easy configuration changes surfaced by our platform team that yield us over $1 million in monthly AI spend.
How Datadog surfaces AI cost savings opportunities
In order for us to identify opportunities for AI cost savings, we needed to first understand the complete composition of our existing AI cloud spend. This meant tracking costs across AI vendors (including Anthropic, Cursor, and OpenAI), their different model offerings, and how API usage mapped back to individual usage and workflows.
We track this data using the AI Costs feature in Datadog Cloud Cost Management. AI cost data supports normalized tags for provider, model name, and token category, helping us identify inputs, outputs, or tokens related to caching and search operations. Every Claude Code request is run through an AI gateway, where the requests are tagged with their team or product of origin. Doing this gives us a more granular view into where our AI costs originate from; for example, we can see if this spend comes from developers working in Claude Code, assistant APIs, Datadog’s agent skills, or other sources of activity. All of our AI cost data is compiled into a dashboard that our AI Developer Experience team uses to identify areas of high spend and cost trends that can translate into savings opportunities.
