AI is moving fast, and it feels like there's a new concept to learn every week. In an effort to actually understand this whole new world instead of just skimming past it, I've been writing ELI5 articles breaking down concepts that show up constantly in AI conversations but rarely get explained simply. This time, a term that gets thrown around a lot without much explanation: the context window.
AI 101 Recap
The context window is the total number of input and output tokens an LLM can consider while generating a response. Your prompt, the conversation history, and even the model's response all share that same "budget" of tokens. As a conversation with an LLM grows longer, more tokens live in this window.
So far so good. Every AI model has a limit on how many tokens it can hold in its "working memory" at once. So the interesting part with the context window is what happens when you reach these limits.
Why are there limits?
