跪拜 Guibai
← Back to the summary

RAG in 162 Lines: The Cognitive Architecture Shift That Turns LLMs Into Scholars

My goal: Use the pyramid expression method to break down layer by layer the design thinking, engineering implementation, and cognitive insights of a complete RAG system.

image.png

image.png

Tip of the Pyramid: Core Conclusion

The essence of RAG is building a retrievable "external memory system" outside the LLM's "internal knowledge." These 162 lines of code prove one thing: RAG is not an engineering trick, but a paradigm shift in cognitive architecture—it transforms an LLM from "a student who can only rely on rote memorization" into "a scholar who knows how to look up information."

Behind this conclusion are three progressively deeper judgments:

  1. RAG solves the problem of information acquisition—from "relying on training memory" to "real-time retrieval of external knowledge bases"
  2. RAG achieves expansion of cognitive ability—the LLM doesn't need to "remember everything," only needs to "know where to find it"
  3. RAG foreshadows the future form of AI systems—a trinity of generation capability + retrieval capability + memory capability

Below, using the pyramid structure, we will break down these 162 lines of code layer by layer, from top-level design to underlying implementation.

Layer 1: The Three Core Pillars of RAG

Although the entire code is only 162 lines, it precisely implements the three core stages of RAG. These three stages are not simple "steps," but three independent subsystems, each solving a fundamental problem:

┌─────────────────────────────────────────────────────┐
│                Three Pillars of RAG                  │
├─────────────────┬──────────────┬────────────────────┤
│  Knowledge       │   Semantic   │    Augmented       │
│  Encoding        │   Retrieval  │    Generation      │
│  (Ingestion)     │  (Retrieval) │   (Generation)     │
├─────────────────┼──────────────┼────────────────────┤
│ Problem Solved:  │ Problem Solved: │ Problem Solved:  │
│ How to turn      │ How to find   │ How to make LLM  │
│ human knowledge  │ relevant      │ generate accurate │
│ into machine-    │ memories in   │ answers based on  │
│ computable       │ semantic space│ external knowledge│
│ semantic vectors │               │                   │
├─────────────────┼──────────────┼────────────────────┤
│ Corresponding    │ Corresponding │ Corresponding      │
│ Code Lines:      │ Code Lines:   │ Code Lines:        │
│ L1-L103          │ L107-L145     │ L148-L162          │
├─────────────────┼──────────────┼────────────────────┤
│ Core Components: │ Core Components: │ Core Components: │
│ Document         │ Retriever      │ ChatOpenAI        │
│ OpenAIEmbeddings │ VectorStore    │ Prompt template   │
│ MemoryVectorStore│                │                   │
└─────────────────┴──────────────┴────────────────────┘

The horizontal relationship of this table is: Knowledge Encoding "stores" the information, Semantic Retrieval "fetches" the information, and Augmented Generation "uses" the information. All three are indispensable.

Below, we delve into each pillar to see what specific problem it solves, how it solves it, and why it is designed this way.

Layer 2: Pillar One — Knowledge Encoding (Document → Embedding → VectorStore)

2.1 What Problem Does This Pillar Solve?

An LLM's training data is "frozen" at a specific point in time. When you ask it "What is the company's latest reimbursement policy," it either doesn't know, or more dangerously, hallucinates. You need a mechanism to turn external knowledge into a form the LLM can "understand."

This leads to the first core problem of RAG: How do you turn a document, a paragraph, or an image into a data structure that a machine can "semantically match"?

The answer is a three-step chain: Document (structured packaging) → Embedding (vectorization) → VectorStore (persistence + retrievability).

2.2 Step 1: The Document Abstraction — An Underestimated Design

new Document({
  pageContent: `Guangguang is a lively and cheerful little boy...`,
  metadata: { chapter: 1, character: "Guangguang", type: "Character Introduction", mood: "Lively" },
})

This is the smallest unit of information in the entire RAG system. The Document has two fields, with completely distinct responsibilities:

Field What it does What it doesn't do Design Intent
pageContent Participates in vectorization (embedding) calculation — Carries the "semantic" part
metadata Filtering, sorting, traceability Does not participate in vectorization Carries the "structured attributes" part

This is a profoundly meaningful separation design. Its core idea is: Semantic matching and structured filtering are two completely different things and should not be mixed in the same vector space.

For example: If you want to find "paragraphs about Guangguang with a lively mood," the correct approach is not to stuff this condition into the search text and hope for the best, but to first filter using metadata { character: "Guangguang", mood: "Lively" }, and then perform semantic retrieval within the filtered results. Metadata enables precise filtering; pageContent enables fuzzy semantic matching—precision and fuzziness each have their own role.

Thought: Why not let metadata participate in embedding? Because metadata values are discrete and definite ("Guangguang" is just "Guangguang"), they don't need "semantic approximation." If a retrieval system needs to determine whether "Guangguang" and "Xiao Guang" are the same person, that's an entity linking problem, not an embedding problem. Mixing orthogonal dimensions into the same vector space only pollutes the semantic signal.

2.3 Step 2: Embedding — The "Translation" from Text to Vector

const embeddings = new OpenAIEmbeddings({
  apiKey: process.env.OPENAI_API_KEY,
  model: process.env.EMBEDDINGS_MODEL_NAME, // text-embedding-v3
  configuration: {
    baseURL: process.env.OPENAI_BASE_URL, // DashScope compatible endpoint
  }
})

What the embedding model does is "translate" a piece of text into a high-dimensional vector. Each dimension of this vector encodes some semantic feature of the text—but what specific feature, humans usually cannot directly interpret. What we can observe is the result: semantically similar texts have vectors that are "close" to each other in space.

There is a noteworthy engineering choice here: Using the OpenAI compatible interface, but actually calling Alibaba Cloud DashScope. This leverages LangChain's ChatOpenAI and OpenAIEmbeddings support for baseURL—they are thin wrappers around the OpenAI SDK, and any service implementing an OpenAI-compatible API can be seamlessly integrated.

The engineering philosophy behind this choice is: No hard dependency on a specific vendor. Change the baseURL, and you can switch from DashScope to OpenAI, Azure, or a locally deployed vLLM—without changing a single line of code.

2.4 Step 3: MemoryVectorStore — "Vector Database in Memory"

const vectorStore = await MemoryVectorStore
  .fromDocuments(documents, embeddings);

What this single line of code does:

  1. Iterates through every Document in the documents array
  2. Calls embeddings to vectorize the pageContent of each Document
  3. Stores the vector and the original Document together in memory

MemoryVectorStore is the most "lightweight" choice here. The code comments state it bluntly:

// In-memory vector store for RAG learning, or lightweight use
// psql
// Help me install PostgreSQL, set root password to 123456, enable vector storage extension

This is an intentional "imperfection." Choosing in-memory storage means:

In the learning and prototyping phase, running through the complete chain first, then replacing underlying components, is the correct engineering rhythm. When you need to go to production, MemoryVectorStore can be replaced with any vector database like pgvector, Weaviate, Chroma, Pinecone—the upper-level code (the way the retriever is called) remains completely unchanged.

This is the value of LangChain's abstraction layer: It lets you first understand "what to do," then decide "how to do it."

2.5 Why Choose "Stories" Instead of "FAQs" as Test Data?

These 7 Documents were not chosen randomly. They collectively tell a complete story—the narrative arc of Guangguang and Dongdong from meeting to becoming lifelong best friends:

Chapter 1: Character Intro (Guangguang) → mood: Lively
Chapter 2: Character Intro (Dongdong) → mood: Warm
Chapter 3: Friendship Begins (Invitation to play soccer) → mood: Encouraging
Chapter 4: Friendship Develops (Training together) → mood: Helpful
Chapter 5: Climax/Turning Point (Winning the match) → mood: Excited
Chapter 6: Resolution (Best friends) → mood: Joyful
Chapter 7: Epilogue (Many years later) → mood: Warm

If the goal was just to test "does retrieval work," 7 FAQs (like "What is the return policy," "How much is shipping") would suffice. But FAQs have a fatal flaw: Each piece of data is isolated and stateless. Searching for "how to return" in FAQs, as long as there's an embedding related to "return," it will hit.

Story data is different:

This is the true value of metadata: it compensates for the "semantic blind spots" of embedding. Embedding is good at judging "what topic is this passage about," but doesn't know "where in the story this passage occurs," "who is speaking," or "what the mood is." This information is lost in the embedding space but preserved in metadata.

Deep Thought: My story itself is a metaphor. Guangguang represents the LLM—naive, talented, but needing guidance; Dongdong represents RAG's retrieval capability—quiet, structured, providing key information at critical moments. As the story says: "A true friend is someone who helps each other and becomes better together." The relationship between LLM and RAG is the same.

Layer 3: Pillar Two — Semantic Retrieval (Retriever + VectorStore)

3.1 What Problem Does This Pillar Solve?

The knowledge has been stored. Now a user asks a question: "How did Dongdong and Guangguang become friends?" How do you find the most relevant ones from the 7 Documents?

If it were traditional search, you'd need exact keyword matching—"Dongdong," "Guangguang," "friends." But this has two problems:

Semantic retrieval solves exactly this problem: Not matching words, but matching meaning.

3.2 Retriever: The "Standard Interface" for Retrieval

const retriever = vectorStore.asRetriever({
  k: 3 // The 3 most similar
});

asRetriever() is an important abstraction in LangChain. It "wraps" the vector database into a standard retriever interface: Input a question (string), output a list of relevant documents (Document[]).

The key to this design is separation of concerns:

When you call retriever.invoke(question), its internal execution flow is:

User Question (string)
    │
    ▼
[Embedding Model] Vectorize the question
    │
    ▼
[Vector Store] Calculate distance between question vector and all document vectors
    │
    ▼
[Top-K Filtering] Take the K documents with the closest distance
    │
    ▼
[Optional: Deduplication, Filtering, Rerank] Post-processing
    │
    ▼
Return K Document objects

The code explains why a retriever is needed:

// prompt for semantic matching
// Provide a retriever, no need for manual prompt embedding
// Convert the vector database into a retriever
// It's a standard entry point: input a question, output the most relevant document list

"No need for manual prompt embedding" points out the core value of the retriever: The caller doesn't need to know "how text is internally turned into vectors, how distance is calculated." You just give it a question, and it gives you answer snippets. This is the most fundamental and important principle in software development—encapsulation.

3.3 Two Layers of Retrieval: invoke() vs similaritySearchWithScore()

The code simultaneously calls two retrieval methods:

// Method 1: Retriever's standard interface
const docs = await retriever.invoke(question);

// Method 2: Directly call the vector store's similarity search
const scoredResults = await vectorStore.similaritySearchWithScore(question, 3);

The comment is candid: // Also want scoring, originally unnecessary.

This reveals a mindset in real development scenarios: Production path + Debugging path in parallel.

retriever.invoke() similaritySearchWithScore()
Positioning Production path Debugging/Diagnostic path
Return Value Document[] [Document, score][]
Includes Score No (by default) Yes
Post-processing May include dedup/filter/rerank Pure vector distance calculation
User Downstream generation stage Developer (checking retrieval quality)

The engineering judgment behind this is: Results for end-users should be concise (just documents), but for developers, they should be transparent (give scores). The value of scores lies in:

  1. Evaluating retrieval quality: If the similarity of the Top-1 result is only 0.3, you should suspect whether the retrieval actually found relevant content.
  2. Diagnosing data problems: If a certain document consistently scores very low, its content phrasing might be ambiguous and needs rewriting.
  3. Adjusting Top-K: Observe the score gap between the K-th and (K+1)-th items to decide whether to widen or narrow the retrieval window.

3.4 The Meaning of Similarity Scores: 1 - score = Cosine Similarity

const similarity = score != null ? (1 - score).toFixed(4) : "N/A"

This line of code performs a mathematical conversion. The score returned by similaritySearchWithScore is cosine distance, ranging from [0, 2]:

By doing 1 - score, we convert it to cosine similarity, ranging from [-1, 1]:

The larger the number, the more semantically similar. This conversion makes the meaning of the score more intuitive. In practice, vectors produced by modern embedding models like text-embedding-v3 usually fall in the positive quadrant, so similarity generally floats between 0 and 1.

Deep Thought: Similarity scores measure not only "what was retrieved," but also "how good is the quality of your knowledge base." If the Top-1 document retrieved has a similarity of only 0.5, it doesn't necessarily mean the retrieval algorithm is flawed—it more likely means your knowledge base simply lacks high-quality matching content. Scores are a diagnostic indicator of knowledge base health.

Layer 4: Pillar Three — Augmented Generation

4.1 What Problem Does This Pillar Solve?

You have retrieved the 3 most relevant document snippets. But the LLM won't automatically read them—you need to "stuff" these snippets into the Prompt, and then tell the LLM: "Please answer the question based on these materials."

This step seems simple—isn't it just string concatenation?—but it is actually the step most prone to error.

4.2 Prompt Construction: The Art of Context Orchestration

const context = docs
  .map((doc, i) => `[Snippet ${i}]\n ${doc.pageContent}`)
  .join("\n\n-----\n\n");

const prompt = `You are a teacher who tells stories about friendship. Based on the following story snippets, answer the question using warm and vivid language. If the story doesn't mention it, say "This detail hasn't been mentioned in the story yet."

Story Snippets:
${context}

Question: ${question}

Teacher's Answer:`;

These 10 lines of code contain several carefully designed details:

First, Snippet Markers: [Snippet 0] [Snippet 1] [Snippet 2]

Why number each snippet? Not for sorting (the most similar is already first), but to allow the LLM to cite information sources when generating an answer. If the LLM says "According to the second snippet...", a human reader can trace the origin of the answer.

The deeper value lies in explainability—users can know which piece of original text the LLM's answer was reasoned from, rather than being fabricated out of thin air.

Second, Separator: \n\n-----\n\n

Two newlines + five dashes + two newlines. This is not arbitrary—it forms a clear "document boundary" marker in the LLM's tokenization. Most LLM tokenizers recognize \n\n as a paragraph break and ----- as a separator line; combined, they form a strong signal: "Below is a completely new information snippet."

Third, Anti-Hallucination Instruction: If the story doesn't mention it, say...

This is one of the most important prompt engineering techniques in RAG scenarios. The LLM's default behavior is to "answer as best as possible"—even without relevant information, it will fabricate an answer based on its training knowledge. When you are doing RAG based on a specific knowledge base, you need it to only answer based on the materials you provide.

This instruction clearly delineates the LLM's "knowledge boundary": use only the materials given to you to answer; if the materials don't have it, explicitly say so. This is a miniature "guardrail."

Fourth, Role Setting: You are a teacher who tells stories about friendship

This isn't just about making the tone warmer. Role setting changes the LLM's output style and narrative perspective. If you don't set a role, the LLM might answer in a cold, encyclopedic tone—"Guangguang and Dongdong's friendship began in kindergarten and deepened through training soccer together...". Adding a role makes it organize language in a warmer way.

4.3 Generation: Everything Converges Here

const response = await model.invoke(prompt);
console.log(response.content);

model.invoke(prompt) is the endpoint of the entire pipeline. From the user's question, to the retrieved documents, to the carefully constructed Prompt, everything finally converges into a single LLM call.

This chain can be understood as a timeline:

t0: User asks "How did Dongdong and Guangguang become friends"
t1: Question vectorization (embedding API call ~50ms)
t2: Vector similarity retrieval (in-memory computation ~1ms)
t3: Return Top-3 documents
t4: Construct augmented Prompt (string concatenation ~0ms)
t5: LLM generates answer (Chat API call ~2000ms)
t6: Return final answer

The bulk of the time cost lies in the two API calls (t1 embedding and t5 generation). The vector retrieval itself (t2) is almost instantaneous at the scale of 7 documents—this is why MemoryVectorStore is perfectly adequate for this scenario.

Layer 5: Cross-Cutting Concerns — Design Decisions Spanning the Three Pillars

5.1 The Significance of temperature: 0

const model = new ChatOpenAI({
  temperature: 0,
  model: process.env.MODEL_NAME,

temperature: 0 means the LLM will produce near-deterministic output for the same Prompt every time. In chit-chat scenarios, a higher temperature (0.7-0.9) might be needed for diversity, but in RAG scenarios:

Thought: Controlling temperature represents a trade-off—between "letting the LLM improvise freely" and "making the LLM faithful to the materials." In RAG scenarios, the scale should tip towards the latter.

5.2 Layered Management of Environment Variables

// .env file
MODEL_NAME=qwen-plus
EMBEDDINGS_MODEL_NAME=text-embedding-v3
OPENAI_API_KEY=sk-ws-...
OPENAI_BASE_URL=https://dashscope.aliyuncs.com/compatible-mode/v1

There's a subtle design here: The LLM and Embedding model use the same set of API credentials but different model names. Both go through DashScope's OpenAI-compatible endpoint, but:

They are two completely different types of models, on two different API endpoints on the network—but through the unified OpenAI-compatible interface, LangChain can operate them as the same kind of "thing." This unified abstraction is a core philosophy of LangChain's design.

5.3 The Hybrid Strategy of CommonJS + .mjs

The package.json in both sub-projects has "type": "commonjs", but the source code files use the .mjs extension. The effect of this combination is:

This is a common pattern in the Node.js ecosystem for handling the "CJS and ESM transition period." Positively, it means you can migrate incrementally—no need to change all files to ESM at once. Negatively, it reflects the historical complexity of Node.js's module system.

Layer 6: From rag-test to rag-demo — A Leap in Cognition

Comparing the code differences between the two directories reveals the key increment from a "theoretical understander" to a "system builder."

6.1 rag-test (94 lines, incomplete): I Know the Steps of RAG

rag-test/src/hello-rag.mjs does:

It has no MemoryVectorStore, no retriever, no retrieval, no Prompt construction, no generation call. Even @langchain/classic is not in its dependencies.

6.2 rag-demo (162 lines, complete): I Understand What Problem Each Step Solves

The 68 lines added in rag-demo/src/index.mjs (+68 lines, from 94 to 162), each line is a key increment:

Added Content Lines Problem Solved
Import MemoryVectorStore +1 Has a container for "storage"
Add @langchain/classic dependency — Has implementation of vector store and retriever
MemoryVectorStore.fromDocuments +8 Document→Vector→Storage, completes knowledge encoding
vectorStore.asRetriever +5 Storage→Retriever interface, completes retrieval abstraction
retriever.invoke +8 Executes retrieval, gets relevant documents
similaritySearchWithScore +7 Gets similarity scores, for quality diagnosis
Result printing & similarity conversion +15 Developer visibility
context concatenation & Prompt construction +10 Documents→Background knowledge→Prompt
model.invoke & output +3 Generates the final answer

68 lines, 8 key increments, completing the full closed loop of the RAG pipeline.

These 68 lines are not just "writing a few more lines of code," but "understanding a few more concepts":

Layer 7: Architecture Panorama

Superimposing all the layers above gives the complete architecture of rag-demo. Represented as a data flow diagram:

                          ┌──────────────────────┐
                          │ Knowledge Encoding    │
                          │ Phase                 │
  [7 Documents]  ──────►  │ pageContent           │
  pageContent             │   → Embedding Model   │
  + metadata              │   → High-dim Vector   │
                          │   → MemoryVectorStore │
                          └──────────┬───────────┘
                                     │
                          ┌──────────▼───────────┐
                          │ Semantic Retrieval    │
                          │ Phase                 │
  User Question ────────► │ embedding(question)   │
  "How did Dongdong      │   → Vector distance   │
   and Guangguang        │     calculation       │
   become friends"       │   → Top-K filter (k=3)│
                          │   → Return 3 Documents│
                          └──────────┬───────────┘
                                     │
                          ┌──────────▼───────────┐
                          │ Augmented Generation  │
                          │ Phase                 │
  Retrieval Results ─────► │ context = doc1 +      │
  (3 Documents)            │   doc2 + doc3         │
                          │                      │
                          │ prompt =              │
                          │   system + context    │
                          │   + question          │
                          │                      │
                          │   → ChatOpenAI        │
                          │   → Final Answer      │
                          └──────────────────────┘

From a data flow perspective, RAG transforms a simple "Question → Answer" dialogue into a four-step information processing sequence: "Question → Retrieval → Augmentation → Answer." Each step enriches the amount of information available to the LLM—retrieval supplements knowledge, augmentation reorganizes expression.

Base of the Pyramid: Insights from 162 Lines

Returning to the core conclusion at the beginning: RAG is not an engineering trick, but a paradigm shift in cognitive architecture.

These 162 lines of code give us five concrete insights:

Insight 1: The barrier to entry for RAG is lower than you think. No distributed systems, no GPU clusters, not even a database needed. 162 lines of JavaScript + two API calls can run a complete RAG pipeline. This means any developer with a Node.js environment can personally experience the full RAG workflow within 30 minutes.

Insight 2: Understanding abstractions is more important than understanding implementations. The separation of Document's pageContent/metadata, the interface definition of Retriever, the vectorization capability of Embeddings—these abstractions form the "skeleton" of RAG. Implementations can be swapped (memory→pgvector, DashScope→OpenAI), but the skeleton remains unchanged. Understand the skeleton first, then swap the muscles.

Insight 3: Debugging signals are the dashboard for system quality. The scores returned by similaritySearchWithScore, the structured fields in metadata, the cutoff point of Top-K—these are not "user requirements," but they are the "vital signs of system health." Without these signals, you cannot know if retrieval is effective or if the knowledge base needs optimization.

Insight 4: Choosing good test data is more important than choosing a good model. 7 Documents of story data expose the true capabilities of a RAG system—cross-document reasoning, temporal understanding, emotion tracking—better than 100 FAQs. Test with data that can "stress test" the system, and you can discover problems before they are exposed.

Insight 5: The ultimate form of RAG is a memory system. What these 162 lines implement is "single-turn RAG"—user asks → retrieve → augment → answer. But the code already plants seeds for the next step: chapter in metadata encodes temporal structure, mood encodes emotional state. Utilizing this "metacognitive" information allows RAG to evolve from "remembering facts" to "remembering understanding and feelings"—that is a true memory system.

Appendix: How to Run

cd rag-demo
pnpm install

# Ensure .env is configured with a valid API Key
# MODEL_NAME=qwen-plus
# EMBEDDINGS_MODEL_NAME=text-embedding-v3
# OPENAI_API_KEY=your-key
# OPENAI_BASE_URL=https://dashscope.aliyuncs.com/compatible-mode/v1

node src/index.mjs

The output will include: vector store info → the 3 retrieved documents → similarity score for each document → the friendship story answer generated based on the retrieval results.

Writing Postscript: This article is written using the pyramid expression method—first presenting the core conclusion (RAG is a paradigm shift in cognitive architecture), then unfolding the three pillars layer by layer (Knowledge Encoding → Semantic Retrieval → Augmented Generation), with each pillar further broken down into specific design decisions and code implementations. The advantage of this structure is: no matter which layer the reader stops at, they have already gained a complete understanding of the layers above. Even without reading the entire article, they can still gain something!!! Notes click here