My AI memory benchmark said 98.3%. The number was true — and worthless.

In my last post I introduced Bastra Recall — an MIT-licensed MCP memory server that gives Claude persistent memory as plain Markdown in a local Obsidian vault. I promised a follow-up on retrieval and benchmarking.

Here it is. It starts with me being wrong.

The 98.3% that meant nothing

Early on, I ran an eval against my real vault: 59 memories, and for each one I used its own trigger phrase as the query. Result:

Recall@1: 98.3% (58/59)