Author(s): Srini Dwarakanathan
Originally published on Towards AI.
This is Part 2 of a series on optimizing OpenSearch for production RAG. Part 1 covered semantic retrieval, meaning vector search with Approximate and exact Nearest Neighbor methods. This part covers lexical retrieval, meaning text-based search, and how it complements the semantic path.
Introduction
Retrieval Augmented Generation (RAG) systems retrieve a set of relevant documents and pass them as context to a Large Language Model (LLM). Part 1 discussed the semantic path, which retrieves documents whose embeddings are near the query embedding in a vector space. This part covers the lexical path, which retrieves documents that share tokens with the query.Lexical retrieval is often described as the older, weaker sibling of semantic retrieval. That framing understates what lexical retrieval contributes. Semantic retrieval is strong at intent and synonymy but weaker at exact identifiers, proper nouns, rare terms, and typo-tolerant partial matches. Lexical retrieval covers exactly those cases, and it does so at low latency with a scoring model that is fully explainable. The two paths are complementary, not substitutes, which is why most production RAG systems run both.The two axes from Part 1 still apply. Recall is whether the relevant documents were retrieved. Latency is how quickly. Every design choice in this article moves along one or both of these axes. This article assumes familiarity with OpenSearch basics such as indices, shards, and mappings; for a refresher, see the References section.
