Organizations building generative AI applications usually compare models the same way: dollars per million tokens. It’s the number on every pricing page, so it becomes the number in every spreadsheet. But production workloads don’t buy tokens. They buy outcomes: a resolved support ticket, a completed research brief, a correct financial summary. Between the pricing page and the outcome sit multipliers the sticker price ignores: how often the model is right, how many tokens it needs to get there, and, for agentic workloads, how many turns it takes, because every turn re-sends the growing conversation.
In this post, we share results from an open-source benchmarking harness that measures those multipliers across OpenAI models on Amazon Bedrock (gpt-5.6-luna, gpt-5.6-terra, and gpt-5.6-sol) and two widely used cost-efficient models on the OpenAI API (gpt-5.4-mini and gpt-5.4-nano). We chose the latter two as the cost-optimized baselines many teams start from, not as like-for-like generational peers, because “we run mini or nano today. Is a newer model on Amazon Bedrock worth it?” is the question we hear most. We focus on three questions:
What does a correct answer cost? Not what a token costs.
