Bottom line: if you want one API key across OpenAI, Claude and Gemini and you're choosing mainly on token cost, put a thin router in front of your app and let it price each request before sending it — with per-model pricing pulled from the same API you're calling, not from three vendor pricing pages you'll forget to re-read. For a two-person startup that buys you one credential, one place to compare cost, and one cheap default model you can override per route.
I build RAG and agent features in Python, and cost control quietly became about a third of that work.
Mostly by accident.
What actually lands on the bill (it isn't the sticker rate)
The per-million-token rate is the number everyone compares, and it's the number that misleads everyone. In a retrieval app the input side dominates: system prompt, tool schemas, six retrieved chunks, a few turns of history, and then a 200-token answer. My production ratio sits near 12:1 input to output, which means a model with a scary output rate can still land lower than one with a friendly output rate and a fat input rate. Retries stack on top of that — every unparseable JSON response that triggers a re-ask pays the full input cost twice, so strict schemas are a cost feature before they're a correctness feature.
