We stopped measuring retrieval quality by how many tokens we could fit into the prompt. When...
We stopped measuring retrieval quality by how many tokens we could fit into the prompt.
When long-context models became available, many of us made the same assumption.
If an LLM can read 128K tokens, retrieval suddenly feels less important. Why spend time carefully selecting documents if the model can simply read everything?
It sounds reasonable.
In practice, it wasn't.
A year ago, models could hold a million tokens in context, roughly 750,000 words or ten novels. The capability is impressive but…
Learn why more tokens hurt LLM reasoning, where low-signal noise comes from, and how reranking, hybrid search, and semantic…
There is a tension at the heart of every enterprise RAG system. Better retrieval recall means more...
The advice in 2026 is settled: chunk your documents, embed them, retrieve top-k, feed those to the...
Why RAG needs context judgment, not just better retrieval Most RAG systems optimize for...
The landscape of Large Language Models (LLMs) is rapidly evolving, with context lengths expanding from a few thousand tokens a…
