I watched a draft miss the exact file span I needed, and the failure was embarrassingly clean: the vector was "close," but the chunk I wanted was buried inside a larger blob. That is the kind of miss that looks acceptable in a demo and annoying in production, because the answer is almost right in the way a blurry photo is almost a portrait.
The mechanic underneath that failure is what dense retrieval people call representational collapse: when you ask one embedding to summarize an entire document, the model picks the dominant mode and discards the long tail. That tail is where the precise sentence lives. So the retrieval system returns the right neighborhood and the wrong house — coherent topically, useless operationally.
The fix is to stop asking one vector to do that work. Split the text before embedding, embed each slice independently, and let retrieval search the slices instead of the document. The rest of this post is what falls out of that decision.
The boundary I chose on purpose
I split embeddings by chunk because the retrieval layer needed a smaller unit than a document. A whole-file vector is too coarse when the system has to answer from a specific path, a specific code block, or a specific rewrite. The chunk is the right compromise: small enough to isolate one semantic mode, large enough to preserve the local context around it.
