---

highlight: a11y-dark theme: channing-cyan

RAG System Evolution (Part 1): An Overview of RAG's Development

From "can retrieve" to "operationalizable," how did RAG evolve step by step?

Guide to this article: RAG is not a complex architecture that appeared suddenly, but evolved gradually in the process of solving real problems. This article will survey stages including Naive RAG, Advanced RAG, Modular RAG, Corrective RAG, Graph RAG, Multimodal RAG, Agentic RAG, and Evaluation & Operations, explaining what technology each upgrade introduced and what shortcomings of the previous stage it addressed.

Building a demo-able RAG (Retrieval-Augmented Generation) is not difficult.

Prepare a few documents, chunk them into small pieces, generate vectors, and save them to a vector database; when a user asks a question, retrieve a few similar fragments and hand them to a large language model to generate an answer.

A dozen lines of code can yield a seemingly decent knowledge-base Q&A system.

The truly difficult part is the questions that follow:

• Why does the system fail to find an answer that is clearly in the documents?
• Why does the model still answer incorrectly even when similar content is found?
• Why are product models, error codes, and proper nouns often not retrieved?
• Why are simple questions fast, but complex ones require multiple queries?
• Why can the model still speak with certainty when the knowledge base has no answer?
• Why does it work great in a demo, but problems keep emerging with real data?

The vast array of RAG technologies that emerged later are essentially all answering these questions.

Why write this series?

The most common difficulty in learning RAG is not unfamiliarity with a specific API, but not knowing the relationship between various technologies.

Just as you grasp vector retrieval, you encounter BM25, Hybrid Search, and RRF; continuing further, you run into Reranker, Query Rewrite, GraphRAG, Self-RAG, and Agentic RAG.

• BM25 (Keyword relevance ranking algorithm)
• Hybrid Search
• RRF (Reciprocal Rank Fusion)
• Reranker
• Query Rewrite
• GraphRAG (Knowledge Graph-based RAG)
• Self-RAG (Self-reflective RAG)
• Agentic RAG

If you just study each term one by one, you easily end up with an ever-growing list of technologies, yet still don't know:

Why is this technology needed, and what problem from the previous stage does it actually solve?

Many resources directly present a complex architecture, but rarely explain why the system evolved step-by-step from the simplest retrieval-generation to routing, error correction, graph retrieval, agents, evaluation, and operations.

This series wants to take a different approach.

We won't organize content by framework APIs, nor will we write a separate article for each technical term. Instead, we will walk through the main stages of RAG's development. Each article will only discuss four things:

1. What problems were encountered at the current stage;
2. What technologies were introduced to address them;
3. What these technologies solved;
4. What new problems they left behind.

The code in the articles will only use a small amount of TypeScript-style pseudocode to explain how data flows. The main thread will always be the evolution of RAG's capabilities.

What problem does RAG actually solve?

During training, large language models store linguistic rules and some world knowledge in their model parameters. This knowledge gives the models powerful general capabilities, but also brings several inherent limitations:

• The model doesn't know corporate policies, internal documents, or personal data;
• New knowledge generated after training ends does not automatically enter the model;
• Updating a single fact cannot be done by simply modifying a specific model parameter;
• The model may generate plausible-sounding but incorrect answers when lacking knowledge;
• It's difficult to explain which source a specific conclusion came from.

One approach is to put all materials directly into the large model's context, but as the volume of knowledge grows, problems like Token (the basic unit of text processing for models) costs, interference from irrelevant information, permission isolation, and content updates quickly arise.

RAG's idea is: don't make the model memorize all knowledge. Instead, when answering a question, first find relevant evidence from an external knowledge base.

Standard Generation:
Question → Large Model → Answer

RAG:
Question → Retrieve Evidence → Add Evidence to Context → Large Model → Answer

The three words in the name RAG correspond precisely to three actions:

• Retrieval: Find evidence related to the question from the knowledge base;
• Augmented: Add the retrieval results to the model's context;
• Generation: Have the model organize an answer based on the question and evidence.

Expressed in the simplest pseudocode:

// First, retrieve relevant evidence from the external knowledge base based on the user's question
const evidence = await retrieve(question);

// Then, hand the question and evidence together to the large model, letting it generate an answer based on the evidence
const answer = await generate(question, evidence);

Therefore, the focus of RAG is not just "generation," but obtaining evidence through retrieval that can support the answer.

If the retrieval stage fails to find the correct content, no matter how powerful the subsequent model is, it can only answer based on incorrect or incomplete context.

Why does RAG keep evolving?

RAG is often likened to letting a large language model take an open-book exam.

But an open-book exam doesn't guarantee a correct answer just because you brought the book into the exam hall. The system also needs to solve:

Where to search?
What method to use for searching?
How to judge if the found evidence is correct?
Should it continue searching if the information is insufficient?
What are the relationships between different pieces of evidence?
When should it stop and refuse to answer?

The development process of RAG is essentially the continuous improvement of the ability to find evidence, organize evidence, verify evidence, and operate the system.

The following stages are not a strict chronological timeline, nor are they mutually exclusive product versions. They are more like a capability evolution map: the system only adds new capabilities when a simple solution fails to solve real problems.

Main Stages of RAG Development

For ease of display, closely related stages are merged in the diagram; below, they will be explained separately following the complete capability roadmap.

Before RAG: Traditional Information Retrieval

Before the emergence of large language models, search engines were already solving the problem of "how to find relevant content from a large number of documents."

Important technologies from this period include:

• Inverted Index;
• TF-IDF (Term Frequency-Inverse Document Frequency);
• BM25;
• Tokenizers and Text Analyzers;
• Retriever + Reader;
• Dense Vector Retrieval.

For Chinese search, Elasticsearch (a distributed search and analytics engine), IK (a Chinese tokenizer), and BM25 belong to this set of traditional retrieval capabilities:

Chinese Documents
→ IK Tokenization
→ Elasticsearch builds Inverted Index
→ BM25 calculates keyword relevance

Traditional retrieval solved "finding documents," but typically did not understand multiple sources and organize natural language answers like a large model does.

The emergence of RAG connected the retrieval system and the generative model:

The retrieval system is responsible for finding evidence; the generative model is responsible for understanding the evidence and organizing the answer.

Stage 1: Naive RAG — Enabling Models to Use External Knowledge

Naive RAG established the minimum closed loop for retrieval-augmented generation:

Load → Split → Embed → Store → Retrieve 
→ Augment → Generate 

It uses document loading, text chunking, Embedding (converting text into vector representations), vector databases, and Top-K (the top K results with the highest similarity) retrieval to place a few relevant fragments into a Prompt and hand them to a large model for answering.

The core problem it solves is:

Allowing the model to read private and new knowledge without retraining.

But Naive RAG often treats "semantic similarity" as roughly equivalent to "able to answer the question." In real data, the two are not always the same thing.

Chunking can break complete semantics; vector retrieval is not good at precise numbering; a fixed Top-K can bring in irrelevant content; when the knowledge base has no answer, the system will still return a few of the most similar fragments.

Thus, the next stage began to systematically optimize evidence quality.

Stage 2: Advanced RAG — Making Retrieval More Accurate

Advanced RAG no longer focuses solely on the vector database, but optimizes the complete process before, during, and after retrieval.

Pre-Retrieval

Through structured chunking, semantic chunking, Parent-Child Retrieval, Metadata, and Query Rewrite, it makes documents and questions more suitable for retrieval.

During Retrieval

Using BM25, Dense Retrieval, Metadata Filter, and Hybrid Search, it leverages both precise keywords and semantic similarity simultaneously.

In Chinese business knowledge bases, a common combination is:

Keyword Branch: Elasticsearch + IK + BM25
Semantic Branch: Embedding + Vector Search
Result Fusion: RRF

Post-Retrieval

Using MMR (Maximal Marginal Relevance), Reranker, deduplication, and Context Compression, it places content more likely to support the answer at the front and reduces repetitive, irrelevant context.

What Advanced RAG solves is:

Not just finding similar content, but finding the evidence that can truly answer the question as much as possible.

But as steps increase, another problem emerges: no matter what the user asks, all questions still go through the same pipeline.

Stage 3: Modular RAG — Letting Different Questions Choose Different Processes

Simple knowledge Q&A can search a vector library, order queries might need to call a business API, statistical questions might be better suited for querying SQL (Structured Query Language), and time-sensitive questions might need an external search.

Modular RAG breaks the complete process into combinable modules:

• Router;
• Conditional Routing;
• Multi-source Retrieval;
• Parallel Retrieval;
• Result Fusion;
• Tool Calling;
• State and workflow orchestration.

The system can first determine the question type, then choose different data sources and processing paths:

// Determine which processing path the current question should enter
const route = classify(question);

// Based on the routing result, select the corresponding knowledge base, database, or business interface to obtain evidence
const evidence = await retrieveByRoute(route, question);

// Hand the evidence returned by the selected path to the large model to generate the final answer
const answer = await generate(question, evidence);

It solves:

No longer forcing all questions down a single fixed pipeline.

However, dynamically choosing a path doesn't mean the evidence returned by that path is necessarily correct. The system also needs to judge if it has made a mistake.

Stage 4: Corrective RAG and Self-RAG — Letting the System Check and Correct Errors

This stage begins to add "checking" into RAG.

Main technologies include:

• Document Relevance Judgment;
• Evidence Sufficiency Judgment;
• Query Rewrite;
• Re-retrieval;
• External Search Fallback;
• Answer Grounding (checking consistency between answer and evidence);
• No-answer Detection;
• Self-Reflection.

The basic logic is:

Retrieve Evidence
→ Judge if evidence is relevant
→ Not relevant: Rewrite query and re-retrieve
→ Insufficient evidence: Continue searching or refuse to answer
→ Sufficient evidence: Generate and check the answer

It solves:

Preventing low-quality evidence from entering the generation stage unchecked.

Of course, the model doing the evaluation can also misjudge, and multiple loops increase latency and cost. Moreover, even if every text chunk is correct, isolated text chunks still struggle to represent complex entity relationships.

Stage 5: GraphRAG — Letting the System Understand Relationships Between Knowledge

Vector retrieval excels at finding semantically similar text, but is not good at answering questions like:

• Which people and organizations are involved in a certain event?
• Through which projects are two companies connected?
• What trend do multiple reports collectively reflect?
• Through how many layers of relationships must a conclusion be reached?

GraphRAG extracts entities and relationships from documents, builds a knowledge graph, and then obtains evidence through graph traversal, community detection, community summaries, and Local / Global Search.

A common combination is:

Vector Retrieval: Find relevant text
Graph Retrieval: Find entity relationships and multi-hop paths

It solves cross-document relationships, multi-hop problems, and global summarization.

The cost is that graph construction, entity disambiguation, and incremental updates are more complex. For simple fact-based Q&A, GraphRAG is not necessarily more suitable than standard retrieval.

Stage 6: Multimodal RAG — Making Knowledge No Longer Limited to Text

Knowledge in real documents does not only exist in paragraphs.

Product manuals contain structural diagrams, financial reports contain tables and curves, scanned contracts rely on page layout, and fault descriptions might be directly annotated on device images.

If only plain text is extracted, this information is likely lost before it even enters the retrieval system.

Multimodal RAG introduces:

• OCR (Optical Character Recognition);
• Layout-aware Parsing;
• Table Extraction;
• Image Captioning;
• Multimodal Embedding;
• Page screenshot retrieval;
• Vision Language Model;
• Page and region-level citations.

It solves:

Retrieving and understanding knowledge in tables, images, scans, and page layouts.

But the cost of multimodal parsing, indexing, and model inference is higher. Faced with open-ended complex tasks, the system also needs to decide for itself what to check first and what to check next.

Stage 7: Agentic RAG — Letting the System Autonomously Complete Multi-step Retrieval

The path for traditional RAG is usually pre-written by developers. Agentic RAG lets the model participate in deciding:

• Whether retrieval is needed;
• Which tool should be used;
• Whether the goal needs to be broken down into sub-questions;
• Whether the current evidence is sufficient;
• Whether to continue investigating based on intermediate results;
• When to stop.

Therefore, it uses Planner, Tool Selection, Multi-step Retrieval, Memory, Checkpointer, Human-in-the-loop, Budget control, and Stop Condition.

The working mode changes from a single retrieval to a loop:

Understand Goal
→ Make Plan
→ Select Tool
→ Obtain Evidence
→ Update Plan
→ Continue or Stop
→ Summarize Answer

It solves open-ended, multi-step, cross-data-source research tasks.

At the same time, system behavior becomes harder to predict, errors can accumulate over multiple steps, and latency and cost increase significantly.

The more complex the system, the less you can judge its effectiveness based on just a few manual queries.

Stage 8: Evaluation and Observability — Proving the System Really Got Better

RAG's answers have a degree of randomness. After changing chunk size, Embedding model, recall count, or Prompt, relying on a "feeling it's more accurate" cannot determine if the system has truly improved.

This stage introduces two types of capabilities.

Evaluation

• Recall@K, Precision@K, MRR (Mean Reciprocal Rank), nDCG (Normalized Discounted Cumulative Gain): Evaluate retrieval results;
• Correctness, Faithfulness, Answer Relevance: Evaluate generation quality;
• Citation Accuracy: Evaluate whether citations are accurate;
• No-answer Accuracy: Evaluate whether the system correctly refuses to answer when information is insufficient.

Observability

• Trace and Span;
• Dataset;
• Experiment;
• User Feedback;
• Failure Taxonomy;
• Latency, Token, and Cost statistics.

Tools like LangSmith (a platform for tracing and evaluating LLM applications) are suitable for introduction at this stage, used for recording call chains, managing test sets, and comparing experiment results.

It solves:

Moving from "tuning parameters by feel" to "validating improvements with data."

However, being able to discover problems is not enough. A long-running system also needs to manage knowledge updates, access permissions, costs, and release risks.

Stage 9: Operationalizable RAG — Enabling the System to Run Long-Term

Operationalizable RAG is not about adding a monitoring platform at the end, but about forming a closed loop for the entire system.

It at least includes:

Data Governance

Incremental indexing, document deletion, data quality checks, index versioning, and Embedding model migration.

Permissions and Security

Identity authentication, Document ACL (Document Access Control List), multi-tenant isolation, sensitive information handling, Prompt Injection protection, and tool permissions.

Reliability and Cost

Caching, timeouts, retries, rate limiting, degradation, and Token and step budgets.

Release and Feedback

Prompt, model, and index version management, quality gates, canary releases, rollbacks, and adding online failure samples to the evaluation set.

Ultimately forming the following cycle:

Data Update
→ Build Index
→ Offline Evaluation
→ Canary Release
→ Online Observation
→ Collect Failure Samples
→ Update Evaluation Set and System

It solves:

Not just making RAG run, but enabling it to improve safely, stably, and controllably over the long term.

The Main Thread of RAG's Evolution

Looking back at these stages, it becomes clear they are not a set of unrelated buzzwords.

Each stage solves the problems exposed by the previous one:

If compressed further, this roadmap can be summarized as:

More Complex Does Not Equal Better

These RAG stages are not a mandatory upgrade checklist that must be fully completed.

If the knowledge volume is very small, Naive RAG might be sufficient; if the business has many product models and error codes, BM25 can be added first; only when cross-document relationship problems genuinely exist should GraphRAG be considered; only when tasks require multiple investigations and tool collaboration is Agentic RAG valuable.

The decision to upgrade should not be based on whether a technology is popular, but should first confirm:

1. Where exactly the current system fails;
2. Whether the new technology can solve this failure;
3. Whether the added complexity, latency, and cost are worth it;
4. Whether the improvement before and after the change can be verified through evaluation.

This is also the principle that subsequent articles in this series will consistently follow:

First find the problem, then introduce the technology.

Next Article: Starting from Naive RAG

The next article will start from the smallest RAG closed loop:

Document Loading
→ Text Chunking
→ Embedding
→ Vector Storage
→ Top-K Retrieval
→ Context Augmentation
→ Answer Generation

We will focus on explaining:

• Why the focus of "Retrieval-Augmented Generation" is retrieval;
• What problems Chunk, Embedding, Vector Store, and Top-K each solve;
• How retrieved evidence enters the Prompt;
• Why a functioning Naive RAG quickly encounters bottlenecks.

Only after understanding this minimal closed loop will the subsequent topics of hybrid search, query optimization, reranking, routing, and error correction make sense in terms of what problems they solve.