How to extract meaning from charts and tables in PDFs | Weaviate

Intro​

If you have ever tried to put a stack of investor decks, scientific papers, or annual reports through a RAG pipeline, then you know the drill: set up an Optical Character Recognition (OCR) or text-extraction step, pick a chunking strategy, embed text, and finally retrieve information. After going through all this, someone may ask a question about revenue in Q2 FY25, and your retrieval would return three pages of unrelated bullet points because the actual answer is in a bar chart that was invisible to your index.

This used to be what people had to do — just leave out the interesting parts of a PDF from RAG such as bar charts with trends, comparison tables, architectural diagrams, and things in general that make PDFs so much more valuable than mere text.

In this article, we'll show you a better way that not only retrieves the rich information embedded in charts, but also eliminates complex processing steps. We'll cover:

Classic RAG: Why OCR and text embedding works great for some (structured and unstructured) data, but not for rich PDFs.