An end-to-end RAG score can show that answer quality changed. By itself, it cannot tell you where to begin investigating. The evidence returned to the model may be wrong, or the model may have mishandled good evidence. One aggregate number hides that distinction and leaves retrieval, reranking, prompting, and generation competing for attention.
I use retrieval relevance and generation faithfulness as separate diagnostic boundaries. They are not an exhaustive taxonomy of RAG failures: corpus defects, bad labels, ambiguous questions, orchestration, context construction, and post-processing can also matter, sometimes at the same time. The practical value of the split is triage. It tells me which boundary to inspect first.
That is why I build the evaluation harness before, or at least alongside, the pipeline. Once the architecture has hardened, it may expose too little to compare what retrieval returned with what generation received and produced. Candidates, ranking, supplied context, and final answers are easier to make observable early than to reconstruct later.
Read retrieval as a scorecard
Retrieval needs more than one number because different metrics expose different shapes of failure. Recall@K asks whether labeled evidence appeared within the cutoff. Precision@K shows how much of that result set is relevant rather than distracting. Mean reciprocal rank (MRR) emphasizes the position of the first relevant result, while normalized discounted cumulative gain (nDCG) evaluates the ranked ordering more broadly.
