RAG Retrieval Accuracy: 38%. After the Fix: 87%. The Model Was Never Touched.

That's a rebuild I shipped. The system: a RAG assistant for fraud analysts — ask it "how do we handle card testing followed by a successful auth?" and it should answer from the team's own SOPs and case history. The complaint: the answers were wrong, therefore the model must be dumb, therefore procurement should buy a bigger model.

The model was fine. It was answering perfectly — from garbage context. Walk the forensic trail with me, because every step is checkable on your own system this week.

Exhibit A: the chunking was destroying meaning before anything was embedded

The ingestion split SOP documents every N characters, mid-sentence. Which means half the vectors in the index encoded fragments like this:

chunk_147 = "...ing to a freight forwarder. In these cases, do NOT"

RAG Retrieval Accuracy: 38%. After the Fix: 87%. The Model Was Never Touched.

Related reading

Your RAG Agent Is Retrieving the Wrong Chunk: 5 Failure Modes We Fix in…

Most RAG Problems Are Retrieval Problems. Here Are 8 Fixes That Worked for Me

Your RAG System Is Lying To You About That Table

Why your RAG accuracy problem is probably stale data (2026)

Your RAG Eval Isn't Flaky. Your Retrieval Is Non-Deterministic.

Why RAG gives wrong answers (and how to fix retrieval failures)