A RAG ask-your-docs chatbot stops hallucinating wrong answers when retrieval is treated as an evaluated evidence pipeline, not a clever prompt.
I build ask-your-docs features in Python, and I start with a blunt rule: retrieve evidence, fit it into the prompt budget, and let the model say "not found" when the evidence is absent. Embeddings and chunking are necessary, but they don't prove that the answerable passage reached the generator. I learned that distinction after watching a RAG demo sound convincing while it cited a neighboring section instead of the one that contained the actual rule.
The practical fix is source-only generation with retrieval evaluation. Measure retrieval recall before judging prose quality, rerank the candidates that survive semantic search, and count tokens before the chat request. The chat model should receive only the selected context and an instruction to decline unsupported questions. Don't use fluent language as a factuality metric.
Tiny change. Big effect.
Why do RAG docs chatbots answer wrong despite embeddings, chunking, and a context window?
