RAG (Retrieval-Augmented Generation) — The Semantic Retrieval Engine

🎯 Why This Module Exists

The RAG (Retrieval-Augmented Generation) module is the semantic retrieval engine that powers every knowledge-based operation in RAGFlow. Without this module, the system would have no way to intelligently search documents, reason about relationships, or synthesize multi-source answers. The RAG module orchestrates a sophisticated pipeline: from multi-format document chunking (14 document-type handlers), through hybrid keyword-semantic retrieval with entity graph reasoning, to hierarchical summarization via RAPTOR. It balances precision (BM25 keyword matching), recall (dense semantic embeddings), and graph-based reasoning to surface the most relevant context. This module exists separately because retrieval is a fundamentally different concern from indexing, generation, and orchestration—each with its own performance bottlenecks and optimization strategies. The entire system's quality depends on RAG relevance; a weak retrieval engine cannot be fixed by a better LLM. — Replace with 150+ words explaining what problem this module solves, why it exists as a separate module (not merged into something else), and what the system would look like without it. Use narrative voice: "Without this module, every component would need to..."

🚶 Step-by-Step Narrated Walkthrough

Follow a single user query to retrieval result as it flows through the entire RAG (Retrieval-Augmented Generation) pipeline:

1. Query arrives at RAG module — User query enters the pipeline. The system determines if it's a search, summarization, or chat request. Typical latency: <1ms.
2. Query encoding — The query is tokenized and embedded using a dense embedding model (e.g., BGE-large). Simultaneously, keyword analysis (BM25 tokenization) prepares it for hybrid retrieval. Latency: ~50-200ms depending on model.
3. BM25 keyword search — Traditional TF-IDF based search against inverted index. Fast O(log n) lookup. Returns top-k keywords matches. Latency: ~5-20ms.
4. Dense semantic search — Vector similarity search against embedding indices (Elasticsearch, Infinity, ONNX). Returns semantically similar chunks even with different keywords. Latency: ~20-100ms depending on index size.
5. Entity graph traversal — If entity relationships exist, traverse the knowledge graph for related concepts and entities. Adds tertiary ranking signal. Latency: ~10-50ms.
6. Result fusion — Combine BM25 scores (40%) + dense scores (60%) + entity signals. Re-rank by combined score. This is where RAG's "hybrid" advantage emerges. Latency: <5ms.
7. Deduplication & filtering — Remove duplicate chunks, filter by confidence threshold, apply any ACL rules. Latency: <5ms.
8. Context window preparation — Truncate or expand results to fit LLM context window. Preserve order and inter-chunk relationships. Latency: <5ms.
9. Optional: hierarchical summarization — If context is too large, use RAPTOR (Recursive Abstractive Summarization over Tree Structures) to compress while preserving key facts. This involves clustering chunks, LLM summarization, and building a summary tree. Latency: ~1-5 seconds (only when needed).
10. Final result to LLM — The combined ranked context list is passed to the generation module. The LLM uses this context to synthesize an answer. RAG's job is complete; now generation takes over. Latency: <1ms for RAG handoff.

RAPTOR: Hierarchical Summarization via Clustering

The Problem: What specific problem does this algorithm solve?

The Approach: How does it work, step by step, in plain English?

Why This Algorithm: What alternatives exist and why weren't they chosen? What are the time/space complexity trade-offs?

graph TD
    A[Input] --> B{Decision Point}
    B -->|Case 1| C[Fast Path]
    B -->|Case 2| D[Complex Path]
    C --> E[Output]
    D --> F[Sub-step] --> E
    style C fill:#e8f5e9
    style D fill:#fff3e0

Hot Paths

• BM25 search: O(log n) inverted index lookup per query term. Typically 5–20ms for 1M-document corpus. Dominates when query is very selective (few matching docs).
• Dense embedding search: O(n) or O(log n) depending on index type (linear for brute-force, logarithmic for HNSW/IVF). 20–100ms for 1M vectors. Dominates when query has semantic shifts.
• Fusion & ranking: O(k log k) where k = number of returned candidates (typically 100–500). <5ms. Negligible compared to search.
• RAPTOR summarization (optional): O(n log n) for clustering + O(log n) LLM calls (tree depth). Only triggered for large result sets. 1–5 seconds but runs offline if possible.

Resource Usage

• Memory: Embedding indices live in-memory (e.g., HNSW graph). A 1M-document corpus with 768-dim embeddings ≈ 3GB. Chunking buffers are temporary (freed after indexing).
• I/O: BM25 index stored on disk (typically 10–20% of chunk data size). Accessed via memory-mapped files. Minimal I/O except during index rebuild (background job via SVR).
• Network: If embedding model is remote (e.g., OpenAI API), adds 500ms–2s latency per query. Consider local ONNX models for privacy and speed.