A few months ago I set out to build a cognitive substrate without a large language model in the answering path. I had a thesis I liked, a Rust workspace, and a lot of conviction.
Then I wrote a three-line baseline that tied it on every metric I cared about.
This is the story of why that was the best thing that happened to the project — and why I'm still building it, just pointed at a sharper target.
The problem I actually care about
When a language model answers a grounded question, it paraphrases its sources. That paraphrase is fluent, often correct, and — this is the part that bothers me — impossible to bind back to its source byte-for-byte. You can cite a document. You cannot prove, after the fact, that the words in the answer are the words that were stored, unaltered, at a known position.
