If you're running LLM-powered features in production, your token bill is probably higher than it should be. Most teams feed the same system prompt, tool definitions, or retrieval context with every request — paying full price to process tokens they've already processed. Prompt caching changes that equation significantly, and it requires almost no refactoring to implement.
What is prompt caching and how does it work
Prompt caching lets you mark a prefix of your prompt as cacheable — system instructions, tool schemas, static documents. The provider stores the attention KV state for those tokens on their side. When your next request starts with the exact same prefix, processing is skipped: you pay only for cache read tokens, which are priced at roughly 1/10th of regular input tokens.
Both major providers support this at the API level. One uses a cache_control block in the request body; the other exposes cache_read_input_tokens in billing data. The mechanics differ slightly, but the principle is identical.
The cost math is straightforward. If you're sending a 10,000-token system prompt with every request and you process 1,000 requests per day, that's 10M input tokens daily. With caching, the first write is slightly more expensive (typically 1.25× input price), but each subsequent read is 10× cheaper. Over 1,000 requests, you pay for 1 write and 999 reads — a 70–80% cost reduction on that prefix.
