BEAM - the Benchmark for Evaluating Agent Memory - is a good benchmark because it tests memory the way production agents actually use it: over very long, multi-session histories, with facts that change over time. It runs at 100K to 10 million tokens across roughly 100 conversations and about 2,000 probing questions in ten task categories, so it cannot be solved by simply enlarging the context window. Instead of one-shot recall, it measures whether an agent can extract what matters, keep beliefs up to date as reality changes, reason across sessions and event order, and still retrieve the right fact after thousands of intervening turns. Older benchmarks like LoCoMo and LongMemEval are close to saturated, while BEAM stays hard - which is exactly what makes it useful for telling memory systems apart.
As soon as you ship an AI agent that is meant to remember things - a user's preferences, a project's history, a customer's past tickets - you need a way to tell whether its memory actually works. The obvious tests turn out to be weak. Ask a model a question a few thousand tokens after the answer appeared and modern models pass easily, which tells you almost nothing about how memory holds up over days, sessions, and changing facts. BEAM was built to close that gap by evaluating memory at the scale and messiness real agents face. This post explains what BEAM is, what it measures, why it is a better yardstick than the benchmarks it is replacing, and where even BEAM stops short. We build and evaluate memory systems ourselves - our memory layer Suprflo is measured against BEAM - so this is the lens we use when we judge whether an agent will remember well in production.
