RAGFlow › deepdoc

Deep Document Understanding (deepdoc)

RAGFlow's neural document parsing pipeline — from scanned PDFs to structured chunks.

Why deepdoc?

Simple text extraction fails on complex documents. deepdoc combines layout recognition, OCR, and table understanding to extract structured information while preserving semantic relationships.

37
Python Modules
13.5K+
Lines of Code
12
Document Types
80+
OCR Languages

Architecture at a Glance

deepdoc orchestrates three layers: file-format specific parsers (PDF, DOCX, Excel, HTML, etc.), neural vision models (layout recognition, OCR, table detection), and semantic chunking that respects document structure.

graph TB subgraph Input["Input Formats"] PDF["PDF"] DOCX["DOCX"] HTML["HTML"] MD["Markdown"] EXCEL["Excel"] end subgraph Parser["Parser Layer"] P1["PDF Parser"] P2["DOCX Parser"] P3["HTML Parser"] P4["Format Converters"] end subgraph Vision["Vision Layer"] Layout["Layout Recognizer"] Table["Table Structure"] OCR["OCR Pipeline"] end subgraph Output["Output Chunks"] Chunk1["Text Blocks"] Chunk2["Tables"] Chunk3["Figures"] end Input -->|format| Parser Parser -->|content| Vision Vision -->|regions| Output classDef inputStyle fill:#1a3a52,stroke:#58a6ff,color:#f0f6fc classDef parserStyle fill:#2d5a3d,stroke:#3fb950,color:#f0f6fc classDef visionStyle fill:#5a2d3d,stroke:#f85149,color:#f0f6fc classDef outputStyle fill:#3d5a2d,stroke:#d29922,color:#f0f6fc class Input,PDF,DOCX,HTML,MD,EXCEL inputStyle class Parser,P1,P2,P3,P4 parserStyle class Vision,Layout,Table,OCR visionStyle class Output,Chunk1,Chunk2,Chunk3 outputStyle

Sub-Modules

Parser Module

File-format specific extraction — PDF, DOCX, Excel, HTML, Markdown, JSON, Text, EPUB, PowerPoint.

  • pdf_parser.py — Master PDF handler (85KB)
  • docx_parser.py — DOCX via python-docx
  • excel_parser.py — XLSX cell extraction
  • html_parser.py — HTML tree traversal
  • Alternative engines — Docling, MinerU, PaddleOCR, TCADp, OpenDataLoader

Vision Module

Neural recognizers for layout analysis and OCR — DLA-based layout detection, ONNX inference, table cell recognition, and multi-language OCR.

  • layout_recognizer.py — 12-class layout detection
  • ocr.py — Text detection & recognition
  • table_structure_recognizer.py — Cell grid decomposition
  • recognizer.py — ONNX base class
  • postprocess.py — NMS & filtering

Narrated Walkthrough: Scanned PDF to Chunks

Let's trace how deepdoc processes a real-world document — a scanned 10-page financial report with mixed layouts, tables, and figures.

  1. Input: scanned_report.pdf (10 pages, 300 DPI, JPEG-compressed pages)
  2. Load PDF: pdf_parser.py opens file via pdfplumber
    • Attempt text extraction per page
    • Pages 1-3: success (OCR'd layer embedded)
    • Pages 4-10: no text found (pure image scans)
  3. Render & Zoom: Pages 4-10 rendered at 3x zoom (900 DPI equivalent)
    • Improves OCR accuracy
    • Helps layout recognizer detect small elements
  4. Layout Recognition: LayoutRecognizer processes each page image
    • ONNX DLA model detects 12 classes: Title, Text, Table, Figure, Caption, Header, Footer, etc.
    • Outputs bboxes: {x, y, w, h, class_label, confidence}
    • Example: page 4 → [Title@top, Text@left_col, Figure@right, Table@bottom, Caption@figure]
  5. Region Assignment: Text from steps 2 & 3 assigned to detected bboxes
    • Text-class bboxes: assign extracted text
    • Figure-class bboxes: crop image, save separately, extract caption from adjacent Text bbox
    • Table-class bboxes: mark for specialized handling
  6. Table Structure Recognition: For detected Table bboxes
    • Crop image region
    • Run table_structure_recognizer → detects cell grid (rows, cols, merged cells)
    • Reconstruct as Markdown table or HTML
  7. OCR Fallback: Pages with low text confidence
    • Run CRAFT text detection on image
    • Run PaddleOCR on detected regions
    • Merge results, preserving spatial order
  8. Page Assembly: Assemble all regions into semantic blocks
    • Example page 4 result: [{type: "title", text: "Q3 2025 Report"}, {type: "text", text: "..."}, {type: "table", html: "..."}, {type: "figure", text: "...", image_path: "fig_1.png"}]
    • Bbox-Aware Chunking: Respect semantic boundaries when chunking
      • Keep tables whole (or split by rows if oversized)
      • Keep figures with captions
      • Split long text blocks by heading hierarchy, not token count
    • Output: List of chunks, each with:
      • Content (text, HTML, or image reference)
      • Type (heading, body, table, figure)
      • Page number & bbox coordinates (enables spatial search)
      • Confidence scores (OCR, layout, table detection)
      • Metadata (figure captions, table row/col indices)

      Design Philosophy

      What is this? — deepdoc's core insight: layout matters. A document is not a flat sequence of characters—it's a 2D spatial arrangement. Respecting this arrangement during parsing preserves semantic relationships that flat text extraction loses.
      Why multiple parsers? — Different documents demand different strategies. A scanned resume needs OCR + text layout. A born-digital business report needs table structure. Chinese documents need language-specific OCR. deepdoc offers pluggable engines so you choose the right strategy for your corpus.
      Why ONNX + PaddlePaddle? — ONNX is framework-agnostic (deploy anywhere, no TensorFlow/PyTorch dependency). PaddlePaddle models were trained on massive document datasets (stronger than general vision models). Combined: portable, accurate, lightweight.

      Key Concepts

      • ConceptExplanation
      BBox (Bounding Box)Rectangular region {x, y, width, height} on a page. Every detected layout element (text block, table, figure) has a bbox for spatial awareness.
      Layout ClassOne of 12 semantic labels: Text, Title, Figure, Table, Figure caption, Table caption, Header, Footer, Reference, Equation. Assigned by LayoutRecognizer.
      OCR Confidence0-1 score from PaddleOCR per character. High confidence = reliable text; low = degraded image or unusual font.
      Table StructureGrid of cells with row/col indices and merge information. Enables round-trip to Markdown/HTML without data loss.
      Fallback ChainOn failure: try faster method first, fall back to slower but more robust method. E.g., text extraction → OCR.
      Semantic ChunkingRespects document structure (headings, table boundaries, figure-caption pairs) when splitting text, unlike token-based chunking.

      Common Patterns

      Pattern 1: Born-Digital PDF with Tables

      Fast path: pdfplumber text extraction → layout recognition → table detection → HTML reconstruction. No OCR needed.

      Pattern 2: Scanned Document (100% image)

      Slow path: render page → layout recognition → OCR for text regions → table detection → assembly. Fallback OCR compensates for missing text layer.

      Pattern 3: Multi-Column Layout

      Layout recognizer detects column-ordered text blocks. Spatial reassembly preserves reading order before chunking.

      Pattern 4: Figure + Caption

      Figure bbox detected, adjacent Text bbox with caption extracted, both linked in output metadata for RAG context.

      Summary

      deepdoc transforms document parsing from "extract all text, hope for the best" to "understand layout, extract intelligently, preserve structure." By combining layout recognition, language-specific OCR, table analysis, and semantic chunking, deepdoc enables RAG systems to work with real-world documents—scanned, digital, mixed-format, anything.

      Generated: 2026-05-06 | Source: /mnt/disk1/zy/rag/ragflow/deepdoc/ | Revision: e8f19aa33