跪拜 Guibai
← Back to the summary

RAG's Full Stack: A 21-Chapter Retrospective and Eight Frontier Directions

📖 Chapter Learning Objectives

  • Quickly locate the chapters in this column that should be referenced based on your own business scenario
  • Understand the technical connotation and applicable scenarios of frontier directions from Graph RAG to on-device RAG
  • Develop a personal learning plan from "knowing how to use RAG" to "being able to contribute to the RAG community"

Introduction: From Building Systems to Looking Ahead

Chapter 20: End-to-End Practice: Enterprise Knowledge Base Q&A System used a complete enterprise knowledge base system to tie together the technical points of the previous 19 chapters, bringing the engineering practice section of the column to an end. The final chapter puts down the code and does two things: review the complete knowledge system you have learned across these 21 chapters, and look at where RAG technology is heading.

I. Full Column Knowledge System Review

The progressive relationship of the three stages can be seen intuitively through the following panoramic diagram.

flowchart LR
    %% Outer main diagram: left to right (LR)

    subgraph Sub1 [Basic Chapters 1-8]
        direction BT  %% Inner subgraph: bottom to top (BT)
        A1[Chapter 1: RAG Concepts and Principles]
        A2[Chapter 2: Text Embedding]
        A3[Chapter 3: Vector Databases]
        A4[Chapter 4: Document Loading]
        A5[Chapter 5: Document Splitting]
        A6[Chapter 6: First Pipeline]
        A7[Chapter 7: Document Q&A Bot]
        A8[Chapter 8: RAG Limitations and Challenges]
        
        A1 --> A2 --> A3 --> A4 --> A5 --> A6 --> A7 --> A8
    end

    subgraph Sub2 [Advanced Chapters 9-15]
        direction BT
        B1[Chapter 9: Retrieval Strategies]
        B2[Chapter 10: Post-Retrieval Processing]
        B3[Chapter 11: Prompt Engineering]
        B4[Chapter 12: Evaluation Systems]
        B5[Chapter 13: Performance Tuning]
        B6[Chapter 14: Advanced RAG Patterns]
        B7[Chapter 15: Agent and RAG]
        
        B1 --> B2 --> B3 --> B4 --> B5 --> B6 --> B7
    end

    subgraph Sub3 [Practical Chapters 16-21]
        direction BT
        C1[Chapter 16: Production Architecture]
        C2[Chapter 17: Performance Optimization]
        C3[Chapter 18: Production Deployment]
        C4[Chapter 19: Monitoring and Observability]
        C5[Chapter 20: Enterprise End-to-End]
        C6[Chapter 21: Review and Outlook]
        
        C1 --> C2 --> C3 --> C4 --> C5 --> C6
    end

    %% Outer connections: use subgraph IDs to connect, maintaining LR flow without breaking internal BT layout
    Sub1 --> Sub2 --> Sub3

(1) The basic chapters build the skeleton of RAG: understanding the principles of vector retrieval, learning to operate document loading and splitting, and running through the first retrieval-generation pipeline.

(2) The advanced chapters dive into optimization of each link: retrieval strategy tuning, prompt design refinement, evaluation system construction, and the introduction of Agent patterns.

(3) The practical chapters integrate the optimization schemes from the previous three chapters into a mass-producible engineering system, covering architecture design, deployment strategy, and monitoring alerting.

Reading paths for different business scenarios can refer to the table below.

Your Need Key Chapters
New to RAG, want a quick overview Chapters 1, 6
Retrieval results are poor, can't find what you want Chapters 8, 9, 10, 13
LLM often fabricates false answers Chapters 8, 11, 14
System too slow, poor user experience Chapters 16, 17
How to prove the RAG system is actually useful Chapter 12
Need to handle complex multi-step reasoning problems Chapters 14, 15
Ready to deploy the system to production Chapters 16, 17, 18, 19

II. Frontier Directions of RAG Technology

The knowledge system above covers the core technologies of current production-grade RAG, but RAG itself is still a rapidly evolving research field. The following eight directions are the frontier topics with the most investment from academia and industry, ordered from underlying architecture to upper-level applications.

1. Graph RAG: From Isolated Text Fragments to Knowledge Association Networks

Traditional RAG retrieves isolated text fragments, ignoring the relationships between knowledge. Graph RAG extracts entities and relationships from documents into a knowledge graph. During retrieval, it not only matches semantically similar text but also follows the relationship graph to find related knowledge nodes.

Typical scenarios: tracking precedent citation relationships in legal documents, inferring component interaction relationships in drug instructions. Microsoft's open-source GraphRAG project is currently the most active engineering implementation in this direction.

2. Ontology-Enhanced RAG: Adding Conceptual Constraints to Retrieval

Graph RAG focuses on "what is the relationship between entity A and entity B," while ontology focuses on "what type is this concept, what properties does it have, and what is it mutually exclusive with." For example, a knowledge graph tells you "aspirin is used to treat headaches," while an ontology tells you "aspirin belongs to NSAID drugs, and common adverse reactions of NSAID drugs include gastrointestinal bleeding."

Ontology-enhanced RAG uses the concept hierarchy and reasoning rules of the ontology during retrieval to expand queries, filter irrelevant results, and discover hidden contradictions. In highly standardized fields such as healthcare and manufacturing, ontology constraints can help RAG avoid dangerous errors like "applying one drug's name to another drug's contraindications."

The relationship with Graph RAG is complementary rather than substitutive. The graph handles instance-level relationships (who/whom), while the ontology handles concept-level constraints (what type/what property). Combining the two can achieve complete knowledge coverage from concrete facts to abstract rules.

3. LLM Wiki: Self-Evolving Knowledge Base

If ontology-enhanced RAG requires manual construction of concept systems, the LLM Wiki approach is completely the opposite: it lets the system automatically build a structured knowledge base from scratch. The core process: the system continuously receives documents and user conversations, the LLM automatically extracts entity and relationship entries from them, organizes them into structured pages similar to Wikipedia, and conflicts between new and old entries are automatically merged or flagged after comparison by the LLM.

A typical application of LLM Wiki is enterprise internal knowledge management systems. Dialogue content generated by employees during daily Q&A can be automatically digested into knowledge entries by the system and become searchable by others the next day. Its relationship with RAG is a two-way enhancement: RAG provides raw material and verification channels for LLM Wiki, and LLM Wiki provides RAG with a more refined and structured retrieval source than raw documents.

4. Multimodal RAG: Unified Retrieval of Images, Text, Tables, Audio, and Video

The RAG systems in this column process text. Multimodal RAG extends retrieval and generation capabilities to images, tables, charts, audio, and video. A multimodal RAG system can take a user-uploaded product architecture screenshot, find the corresponding architecture description from the document library, and give an answer with text plus annotated images.

Between 2024 and 2026, multimodal models represented by Claude Sonnet 5 have been able to simultaneously understand images, tables, and text in a single call, greatly lowering the implementation threshold for Multimodal RAG—no longer requiring separate image description generation and OCR steps, but directly using vision-language models for end-to-end understanding of mixed-format retrieval results.

5. Agentic RAG: From Tool Invocation to Autonomous Research

The Agent described in Chapter 15: Agent RAG is the starting point of Agentic RAG. The evolution direction of Agentic RAG is from "the user says one step, do one step" to "the Agent autonomously plans multi-step reasoning paths," and then to the most cutting-edge Deep Research mode in 2026—after receiving an open-ended question, the Agent autonomously executes tens of minutes of multi-round retrieval, reading, summarization, and cross-verification, ultimately producing a research report with complete citations.

This mode is changing the evaluation criteria for RAG systems. Traditional RAG metrics are MRR and faithfulness. Deep Research also needs to evaluate the breadth of information coverage (whether important opposing viewpoints are missed), the depth of reasoning (whether it is only superficial summarization rather than causal analysis), and the proactivity of updates (whether it discovers contradictions between retrieval results and actively seeks a third source to arbitrate).

6. Federated RAG: Privacy-Preserving Cross-Organization Retrieval

Current enterprise RAG faces a dilemma: a single company's document volume is not enough to cover complex cross-domain problems, but directly sharing documents with third parties involves privacy and compliance risks. The core idea of Federated RAG is "data does not leave the domain, retrieval crosses organizations." Each participant maintains an independent vector index on its own private server. When a query is made, a coordination node sends retrieval requests to all participants simultaneously, and each party only returns the most relevant document fragments without exposing the full index.

The technical difficulties of this scheme lie in deduplication and global ranking of retrieval results (each party only knows its own top-K without a global perspective), as well as preventing malicious participants from constructing special queries to reverse-infer the private document contents of other parties. Currently, this direction is mainly driven by organizations in medical alliances and financial compliance fields.

7. Small-Model On-Device RAG: RAG Is Not Only in the Cloud

With the maturity of lightweight models such as Phi-4, Gemma 3, and Llama-4, as well as GGUF/ONNX quantization schemes, running a complete RAG pipeline locally on phones and laptops has become possible. Application scenarios for on-device RAG include fully offline personal knowledge assistants, consulting local technical documentation on airplanes, and scenarios involving highly sensitive information that cannot be processed in the cloud.

The bottleneck of on-device RAG is not retrieval (local vector index query speed is fast) but the generation stage—lightweight models still lag behind cloud large models in citation annotation and logical reasoning. A practical compromise is local retrieval plus cloud generation: the vector index of sensitive documents stays on the device, and after retrieving relevant fragments, if the question does not involve sensitive content, the fragments are sent to a cloud model for answer generation.

8. RAG in the Long Context Era: Fusion Rather Than Replacement

With the emergence of 128K, 200K, and even 1M context window models, a natural question is "do we still need RAG?"

The answer was already explained in the three-scheme comparison in Chapter 1: Understanding RAG: Long Context solves deep understanding of a single long document, while RAG solves efficient positioning of massive dynamic knowledge. The two are not a replacement but a fusion. Use RAG to filter the most relevant fragments from a knowledge base of 10 billion tokens, and use Long Context to let the model synthesize reasoning within a sufficient context window. As a new generation of models integrates long context and retrieval capabilities at the API level, this fusion trend will further accelerate.

III. Continuous Learning Roadmap

The path after completing 21 chapters is shown as a reference route in the diagram below.

graph LR
    A[This column<br/>21 chapters] --> B[Framework deep dive<br/>LangChain/LlamaIndex]
    A --> C[LLM basic theory<br/>Transformer/Attention]
    B --> D[Open-source project contribution<br/>LangChain/Chroma]
    C --> D
    D --> E[Self-developed RAG system<br/>implement from scratch without framework]
    E --> F[Publish technical articles<br/>share frontline experience]

Three supplements to this roadmap. First, do not stop at frameworks. The APIs of LangChain and LlamaIndex are constantly changing, but the underlying principles of RAG are relatively stable. After completing this column, try to implement RAG from scratch without LangChain, using only fetch to call APIs plus your own cosine similarity calculation. Second, focus on papers rather than tutorials. RAG is an active research field, and tutorials naturally lag behind the latest progress by 6 to 12 months. Get into the habit of browsing ArXiv weekly, and at least read the core contributions of a paper by looking at the figures. Third, accumulate experience with real projects. Every business scenario has its own document characteristics, knowledge structure, and user habits, and there is no universal one-size-fits-all parameter. Migrate the methodology of this column to your specific business and continuously iterate using the evaluation system in Chapter 12.

FAQ

Q1: What else do I need to learn after completing this column

If the goal is to become an expert in the RAG field, the next step is to dive deeper into the LLM itself. For example, understand the Transformer architecture, attention mechanisms, and Tokenization principles. This knowledge will help you make more accurate judgments at every step of the RAG pipeline, such as why Embedding models perform poorly on certain languages and why instruction-following ability varies between models. It is recommended to start with Jay Alammar's "The Illustrated Transformer" and then gradually dive into papers.

Q2: Will RAG be replaced by better long-context models

It will not be completely replaced, but RAG's positioning will shift from "compensating for insufficient context windows" to "managing retrieval efficiency for massive dynamic knowledge." Even if a model can read 1M tokens at once, finding the exact 1M tokens from a knowledge base of 10 billion tokens still requires retrieval technology. The relationship between the two is complementary. See the three-scheme comparison in Chapter 1 for specific discussion.

Q3: Which should I choose, LangChain or LlamaIndex

This column primarily uses LangChain because its design philosophy is straightforward. LlamaIndex invests more deeply in data management, especially in data connectors and index structures, while LangChain is stronger in the flexibility of Agents and Chains. It is recommended to try both briefly and choose the one that better matches your tech stack and aesthetic. Neither is a lock-in choice, and you can switch at any time.

📚 Extended Reading


Thank you for reading this far.

The journey of 21 articles is not short. If you have practiced from the first article all the way here, you should now have the complete ability to independently build a RAG system. RAG technology develops quickly, but the methodology of understanding principles as a foundation and using engineering practice as a test will not become outdated.

Wishing you build good RAG systems.