Memory & State For AI Agents
Building an AI agent can be tricky. Keeping it on track over a six-month deployment is incredibly hard.
LLMs are stateless by design. Every call starts from scratch, with no memory of what came before. Early agent developers worked around this by dumping the entire conversation history into the context window and hoping for the best.
By now, we know that approach breaks down fast. Latency spikes, and the model’s ability to actually use what’s in context degrades: relevant facts get buried, and when two versions of a fact are both in the window, there’s no guarantee it picks the current one. Token costs balloon too, though prompt caching has softened that blow for stable prefixes. The fix isn’t a bigger context window; it’s treating memory and state as deliberate architectural decisions, not afterthoughts.
Before getting into the patterns, it’s worth being precise about what those two terms mean, because they’re easy to conflate.
