Retrieval-augmented generation is the most requested AI feature and the most commonly botched. The failure is almost never the generation step -- it's retrieval quietly returning the wrong context, and the model confidently summarizing garbage. This checklist comes from building research and analysis pipelines where wrong retrieval means wrong business decisions.
Chunking: respect the document's own structure
Fixed 512-token windows sliced mid-sentence are the default and the mistake. Chunk on semantic boundaries -- headings, paragraphs, table rows -- and prepend each chunk with its breadcrumb ("Annual Report 2025 > Risk Factors > Currency"). A chunk that can't be understood alone can't be retrieved reliably.
Hybrid search beats embeddings alone
Dense vectors miss exact identifiers -- SKUs, names, error codes -- that keyword search catches trivially. Run BM25 and vector search in parallel and fuse results (reciprocal rank fusion is fine). This one change fixes a huge share of "the answer was in the corpus but retrieval missed it" tickets.
