Same query. Same documents. Same model. And the RAG eval can still hand back a different...
Same query.
Same documents.
Same model.
And the RAG eval can still hand back a different Recall@8.
Not because the model is flaky. Because of an ORDER BY clause.
Your agent is non-deterministic and you know it. So is your model-as-judge eval. Here's how to measure judge flakiness, treat…
If your RAG app sometimes answers from the wrong document even though the right one was in your...
You picked a model. You built a RAG pipeline or an agent loop. You ran evals, the results looked...
Retrieval Augmented Generation, or RAG, has become the default way to ask questions about long...
Your retriever returned the right documents. The similarity scores look fine. The answer is still...
Originally published at vivekpatil23.hashnode.dev The Problem With Naive RAG Nobody Talks About Most...
