RAG Takes One Step Further - Using Graph RAG and OCR to Understand Complex Documents

TL;DR OCR turns text inside scanned or image-heavy documents into searchable text, while Graph RAG uses relationships between documents and entities. This helps with reports, contracts, and incident records where simple keyword search misses the context.

In the previous article, we compared RAG to a librarian. The librarian finds materials related to the user's question and hands them to the LLM. But what happens if the book that arrives at the library has no text at all, only images and tables? What if the materials are part of a complex event where many documents are connected to each other?

This article looks at the next two walls RAG runs into, and how OCR and Graph RAG help cross them.

How Graph RAG and OCR Connect

OCR extracts text and table structure from documents, while Graph RAG connects the relationships between extracted entities. Vector search and answer generation sit on top of that flow.

The Limits of RAG That Only Finds Text

The RAG we discussed earlier splits documents into small chunks, converts them into meaning-based numbers called vectors, and stores them. When a user asks a question, it finds chunks with similar meaning and sends them to the LLM.

The problem appears in two situations.

1. When the text is inside an image

Scanned PDFs, photographed contracts, quarterly reports full of tables. If text cannot be extracted, there are no chunks to split. It is like the librarian saying, "This book is only photos, so I cannot read it."

2. When materials are connected to each other

Consider a question like, "A similar process is happening at the site where an incident occurred last year. Which safety rules apply?" Incident records, process standards, and regulations are separate sources. If they stay separate, the answer does not come together. The librarian can find one book at a time, but may not connect the relationships between them.

OCR - A Librarian That Can Read Text Inside Images

OCR solves the first wall. OCR stands for optical character recognition. It extracts text from images.

Modern OCR is not only about reading letters. The important part is recognizing table structure, paragraph position, and the hierarchy between headings and body text. This makes a large difference in fields that contain many dense tables, such as credit review documents, construction specifications, and medical imaging reports.

Once OCR extracts the text, the rest flows into the ordinary RAG process: split by meaning, convert into vectors, store, and search when a question arrives.

Graph RAG - A Librarian That Sees Relationships Between Materials

The second wall is Graph RAG. As the name suggests, it combines a graph database with RAG.

If ordinary RAG finds "chunks with similar meaning," Graph RAG also stores relationships between materials.

For example, suppose there is an incident called "2024 utility tunnel work incident." That incident may be connected to the following items.

• Work process: "work at height"
• Applicable regulation: "occupational safety regulation"
• Safety rule: "mandatory fall-prevention harness connection"
• Similar past incident: "2021 field incident"

When those relationships are tied together as a graph, a single question such as "What should we check for today's utility tunnel work at height?" can bring all four materials together. The LLM can then answer at once: which regulation applies, which checks are needed, and which similar incident should be considered.

The value of Graph RAG is that users no longer have to find materials separately and connect them in their heads. The system already knows the relationships and brings related materials together.

Where It Is Used

OCR and Graph RAG shine in fields where materials are image- or table-heavy, and the materials are strongly connected to each other.

Finance - credit review and market analysis Business plans, financial statements, and market reports can span dozens of pages. OCR brings tables and charts into text, and Graph RAG connects companies, industries, and regulations, so a person in charge can receive the key points from one question.

Industrial safety - field guidance Government safety guidelines, internal specifications, and past incident collections are often used together. When a field manager asks what to check for today's utility tunnel work at height, the system can immediately combine regulations, rules, and similar cases.

Closing

RAG is not a technology that becomes complete all at once. It moves step by step from finding text, to reading information inside images, to connecting relationships between materials.

Minisoft operates this staged RAG approach in finance and industrial safety domains, and we keep experimenting every day with ways to make the librarian smarter.

References

• From Local to Global: A Graph RAG Approach to Query-Focused Summarization
• Microsoft Research: Project GraphRAG
• Azure AI Document Intelligence overview