I Tried Building LLM Memory the Stupid Way — It Worked Surprisingly Well

This started as a fun experiment.

I've been working on a local RAG system , and at some point I started wondering about something that sounds simple but is actually quite difficult:

Can an LLM remember old conversations without sending its entire history back into the context window?

I didn't want to build a complicated memory architecture just to answer that question. So I tried something almost ridiculous. I exported my old chats and extracted conversations from Json files.

The entire memory extractor was basically one rule

I Tried Building LLM Memory the Stupid Way — It Worked Surprisingly Well

Related reading

I built a local-first AI memory layer for LLMs in Rust (no cloud, no API keys)

I Built a Memory Layer for LLM Agents That Knows Which Facts Go Stale

AI Memory Is Broken. Here's What's Finally Starting to Fix It

I Built a Memory API That Beats Mem0 on LongMemEval Without Using a Single LLM…

I built an RL-native memory engine for LLM agents (every memory decision…

The Model Doesn't Remember. You Do