A common objection to agent memory is that you don't need it: context windows are huge now, so just put the whole history in the prompt. We wanted a real answer, not a vibe, so we ran two public long-term-memory benchmarks against a full-context baseline. Here's what we found — including the case where the baseline wins.
The setup
We compared two configurations on the same questions. The full-context baseline stuffs the entire conversation history into the prompt. Eidentic memory ingests the history into its four-tier engine and retrieves only what each question needs. Both use the same model and the same LLM judge. We ran the full sets — no sampling — and we're publishing wins and losses together.
LongMemEval: memory wins across the board
LongMemEval uses long histories — roughly 115k tokens across ~50 sessions, 500 questions. This is where memory should help, and it does: 55.2% overall vs 41.0% for full context, a 14.2-point gain, winning all six question types.
