Your agent feels fast during a demo. Then a real session reaches twenty turns, several tools have returned large payloads, and every response starts taking longer and costing more.
This pattern is often called token drift: the effective input context grows as an agent carries more conversation history, tool output, retrieved documents, and state into each model call. The model itself is not gradually becoming less efficient. The application is asking it to process more material on every turn.
Token drift is manageable, but only if context is treated as a budgeted system resource rather than an unlimited transcript.
What Token Drift Actually Means
Most conversational agents build each request from several sources: a system prompt, tool definitions, recent messages, retrieved context, durable memory, and sometimes a summary of older work. Whether the application sends that state on every request or a provider manages part of it, the model still has an effective context to process.
