I run a multi-agent setup where ten agents analyse the same input. Same document, same market data, same everything. The only difference between them is persona: each one is instructed to look at the material through a different lens.
Ten agents, one shared context. That should be the ideal case for prompt caching. Send the expensive context once, pay full price for it once, and let the other nine reads come back at a fraction of the cost.
My cache hit rate was zero percent on three of the five models I was using, and under seven percent on the other two.
I had been reading the bill for a while and optimising the wrong thing. Here is what was actually happening, because the mistake is structural and I doubt I am the only one making it.
What prompt caching actually matches on
