I was paying full price for input tokens I was sending over and over. A large system prompt, a fixed tool list, the same reference docs on every request. Prompt caching should have made those cheap, except I had three silent bugs that meant nothing was actually caching. Here is what I found when I finally checked the numbers, and how I got my hit rate from zero to consistent.
The one rule that explains everything
Prompt caching is a prefix match. Any byte change anywhere in the prefix invalidates everything after it. The cache key is the exact bytes of the rendered prompt up to each breakpoint.
Render order is fixed: tools, then system, then messages. So your most stable content has to physically come first, and anything that changes per request has to come last. Get the ordering right and caching mostly works for free. Get it wrong and no amount of cache_control markers will save you.
How I knew it was broken
