What is a sparse embedding ? The word sparse means thinly scattered or occurs in a small amount over...
What is a sparse embedding ?
The word sparse means thinly scattered or occurs in a small amount over a large area. Sparse embedding(shortly as S.E) will have a vocabulary(dictionary of words). Words will be stored in a ordered list format.
Basic S.E methdology
Lets assume that vocabulary has 10,000 words. For the given chunks, it will first start to tokenize each of the chunk.
Ex: Chunk 1 -> Redis is a inmemory database
Sparse means thinly spread, scattered, or not dense. In sparse embeddings, chunks are converted into...
In the previous post, we saw some basic methodologies under sparse embeddings. In that, term...
Dense embedding have continuous numeric values. i.e after decimal point values will be present. Chunk...
Key Takeaways RAG pipeline chunking strategies determine retrieval quality more than the embedding...
Chunking sounds like the boring part of RAG. It is also where a lot of retrieval quality is won or...
A short learning path from a weekend project: I indexed my personal markdown notes (~800 chunks),...
