My first real LLM bill surprise wasn't dramatic. No infinite loop, no viral spike. A retry policy I'd written months earlier met a flaky endpoint, and a background job quietly re-sent the same long prompt all night. The bill was just... 40x normal. Nothing "failed", so nothing alerted.
I've spent the last few weeks building a hard spending cap for LLM calls, and most of what I learned wasn't about caps at all. It was about how many small boring ways money leaks when the meter only exists on the provider's side. Here's the list I wish someone had handed me.
1. Billing dashboards are rear-view mirrors
Every provider dashboard answers "how much did you spend?" hours later, after aggregation. Nobody answers "how much are you spending right now, from which feature?" By the time the dashboard shows the leak, the leak already happened. If you take one thing from this post: put the meter inside your process, at the call site, where it can act before the request instead of reporting after it.
2. Total spend is a useless number
