I'm building a small, fully-local research assistant: a RAG over my own papers, running on Ollama, nothing leaving the box. The risk that actually worries me isn't speed or cost. A research tool that cites a wrong number while sounding sure of itself is worse than no tool, because you'll believe it.
Andrej Karpathy's llm-wiki note had a piece I kept thinking about. Instead of re-retrieving from scratch each query, you have the model build a persistent wiki, and during ingest a lint pass checks the pages against each other for contradictions. I wanted something adjacent at answer time: after the RAG drafts an answer, break it into claims and check each against the sources, then flag whatever a source doesn't actually support.
I should be precise about what that is, since the post is partly about citation accuracy. Karpathy's lint compares wiki pages to each other during ingest. What I built compares each answer-claim to its retrieved passage at answer time. That's groundedness (or faithfulness) checking, the same family as the RAGAS faithfulness metric and various self-check methods. The idea to bolt it on came from llm-wiki; the mechanism is standard groundedness checking, run locally on a small model.
