Your agent's memory layer will not throw. It returns three plausible looking chunks, the model answers confidently from them, and nobody notices for a week. That is the real failure mode of LLM memory in production: retrieval quality drifts while every dashboard stays green, so the only defence that actually holds is asserting on what came back before the model ever sees it.
Here is where memory breaks, what the benchmarks say happens at scale, and the verification hooks I wire around retrieval so the failure gets loud.
The silence before the failure
Start with the distinction most teams collapse. Context is what you put in the prompt this turn. Memory is what you can pull back on turn four hundred, in a session that started three weeks ago. Context is a buffer. Memory is a retrieval system, and retrieval systems fail differently from buffers.
A buffer fails visibly. You blow the window, the API returns an error, you see it in logs. A retrieval system returns something no matter what. Ask it for what the user said about their billing preference and it will hand you the nearest neighbours in embedding space. If nothing relevant exists, the nearest neighbours are still returned, just with lower scores that nobody is reading.
