Our agent's auto-compact was supposed to protect the context window. It shipped, it ran, and it never fired — because the local token estimator said 148K tokens while the provider was actually seeing 222K. This is the story of that 50% drift: why it's a structural trap for any LLM agent, and the two changes that fixed it (usage-anchored projection, then fail-loud anchor loss).
If you run an agent that feeds a growing conversation into an LLM, you have some version of this problem: how do you know when the context window is about to blow? The honest answer is that you don't — not locally. Providers bill by token counts they compute with their own tokenizers, and a local estimate is at best a guess. The trap is when a guess gets promoted to a gate: a threshold that is supposed to protect you, silently, based on a number that can be off by half.
That's exactly what happened to us, and the fix ended up being a one-line insight that reshaped the whole feature: when the provider gives you the real number, anchor to it — and if you ever lose the anchor, fail loud.
The failure: a safety net that never fired
Our daemon has an auto-compact feature: before each LLM round, it estimates the conversation's token count, and if the projection exceeds a threshold (a percentage of the configured context window), it summarizes the history to free up space.
