Retrieval Augmented Generation, or RAG, has become the default way to ask questions about long documents. You do not train a model on your data. You just fetch the right pieces of text and hand them to the model at query time. It works well for plain text. It gets much harder when your document is full of tables, and recent work in the field is starting to admit this out loud.
A 2026 benchmark called REAL-MM-RAG built specifically around table heavy documents found that current retrieval models have real trouble understanding tables well, even when the surrounding text is handled fine. That gap is not a rare bug. It shows up constantly in real systems once tables get long or messy.
I want to talk about one specific case. Think about a long report full of tables. Some tables run across two or three pages. The header row is on page one. The numbers continue on page two. And often, the real explanation of what a number means sits far away from the table. It might be a small note at the bottom of the page. It might be a sentence in a paragraph three pages later.
A human reader connects these pieces without effort. A RAG system often cannot.
The chunking problem
