I run a multi-provider LLM gateway in production (OpenAI, Anthropic, Google, DeepSeek and a dozen others behind one endpoint) with prepaid, per-token billing. Getting the metering correct took more iterations than the entire proxy itself. Here is what I wish someone had told me.
One request is never one price
A single chat request produces up to six differently priced components: text input, cached input, cache writes, output, reasoning tokens, and tool units (web search calls, image generations). We ended up writing one charge row per component, each with its own rate, summed per request trace. Every attempt to shortcut this with a single "tokens x price" row broke on the next provider quirk.
Streaming usage is scattered, not at the end
If you only read usage from the final SSE chunk, you will undercount. OpenAI sends usage in the last chunk when you ask for it via stream_options. Anthropic reports input and cache tokens in message_start and the rest in message_delta. The robust pattern: accumulate every usage payload seen anywhere in the stream, keep cached and uncached input separate end to end, and only price the sum at the very end.
