Your RAG system did not fail because the model hallucinated.
It failed because the only “facts” the model saw were a mangled PDF table, an outdated policy, a chunk with missing context, and three near-duplicate paragraphs that pushed better evidence out of the top-k results.
The LLM was downstream of a retrieval pipeline that had already distorted reality.
This is the part of RAG that is easy to miss. Teams spend a lot of time choosing models, tuning prompts, and debating context windows. But in production, a shocking number of failures happen earlier: during ingestion, chunking, indexing, filtering, ranking, and query transformation. By the time the LLM receives the retrieved context, the answer may already be impossible.
The retrieval pipeline is not a neutral search layer. It decides what the model is allowed to know.
