← Back to the summary

A TypeScript RAG Stack That Turns Project Decisions Into a Queryable Asset

This is the first article in the "Building a Private RAG Knowledge Base from Scratch" column. This column will not just stay at the conceptual introduction level; instead, it will use the TypeScript project CorpRAG as the main thread to gradually build a private knowledge base based on LangChain, Ollama, LanceDB, and MCP.

Foreword: Why I Want to Build a Private Knowledge Base

In a long-term iterative project, what is truly easy to lose is often not the code, but the context behind the code.

For example:

Why was JWT ultimately chosen for user authentication instead of Session?
Why was a certain table split instead of continuing to add fields?
Who proposed a certain business rule, and what problem did it solve at the time?
What are the limitations of a certain technical solution, and what are the alternatives?
Why was a pitfall encountered half a year ago stepped into again half a year later?

This information might be scattered across Markdown files, meeting minutes, chat logs, requirement documents, and developers' memories. As a project grows and personnel changes, relying solely on directory categorization and filename searches makes it difficult to quickly find the truly needed answers.

Therefore, what I want to build is not merely a "document management tool," but a private knowledge base that can be queried using natural language.

This knowledge base primarily stores two types of content:

Technical Decisions: Architecture selection, component trade-offs, interface design, performance solutions, incident post-mortems, etc.
Business Decisions: Rule changes, process design, requirement backgrounds, boundary conditions, solution conclusions, etc.

When I ask:

Why did the EXT system choose this specific deployment plan at the time?

The system doesn't just match the words "deployment plan"; it finds the most semantically relevant snippets from existing decision documents and returns the sources along with them. Subsequently, a large language model can be used to organize an answer based on these materials.

This is the goal this column aims to achieve:

Transform scattered project knowledge into a searchable, traceable, and sustainably maintainable personal knowledge asset.

1. What is RAG?

RAG stands for Retrieval-Augmented Generation.

The simplest understanding is:

Before a large language model answers a question, it first searches for information in a designated knowledge base, then answers based on the found information.

A standard large language model primarily relies on the knowledge learned during its training phase. It usually doesn't know about our private projects or the business decisions the team just made. If asked directly, the model might answer "I don't know" or give an answer that sounds plausible but doesn't actually match the project's situation.

RAG adds a retrieval process between the user's question and the large language model:

User Question
   ↓
Retrieve relevant snippets from the private knowledge base
   ↓
Pass "question + relevant snippets" to the large language model
   ↓
Generate an answer based on the project materials

If we compare a large language model to a person skilled at reading and summarizing, then RAG is like placing the materials related to the question on their desk before they answer.

In one sentence:

RAG = Search for information first, then organize the answer.

2. What Problems Can RAG Solve?

1. Enabling Large Language Models to Use Private Knowledge

Project documentation, company policies, product manuals, and internal decisions typically do not appear in a model's training data. RAG allows the model to temporarily read these materials without retraining.

2. Reducing Model Hallucination

Large language models are very good at organizing language, but they don't guarantee that every sentence is factual. RAG uses the retrieved information as the basis for the answer and instructs the model through a prompt to "only answer based on the provided context," thereby reducing unfounded speculation.

It's important to note that RAG can only reduce hallucination, not guarantee its complete elimination. Document quality, retrieval accuracy, prompt constraints, and model capability all affect the final result.

3. Making Answers Traceable

A trustworthy knowledge base should not only return an answer but also the source of the answer, for example:

Conclusion: The EXT system adopts Plan A.
Source: data/EXT-FADR.md

With the source, users can go back to the original document for further confirmation, rather than just trusting a piece of text generated by the model.

4. Reducing the Cost of Knowledge Lookup

Traditional keyword search requires the user to know which words appear in the document. Vector retrieval focuses on semantics; even if the query and the original text don't use the exact same wording, relevant content can still be found.

For example, if a user asks "how is the login state saved," the original text in the knowledge base might be "the system uses Redis to persist session information." The keywords in the two sentences are not entirely consistent, but their semantics might be very close.

5. Avoiding Frequent Model Fine-tuning

If the goal is simply to make the model aware of frequently changing business materials, there is usually no need to retrain or fine-tune the model every time. Updating documents and rebuilding the index is often simpler, cheaper, and easier to track for content changes than retraining the model.

3. How Does a Complete RAG System Work?

RAG can be broken down into two core phases.

1. Indexing Phase: Putting Materials into the Knowledge Base

The indexing phase is typically executed after documents are added or modified:

Raw documents (Markdown / PDF / Word, etc.)
                 ↓
            Load Documents
                 ↓
            Clean Content
                 ↓
            Text Chunking
                 ↓
         Embedding to Generate Vectors
                 ↓
  Write text chunks + vectors + metadata to the vector database

Why is text chunking necessary?

Because a complete document might contain multiple topics simultaneously. If a single vector is generated for the entire document, the semantic meaning expressed by the vector would be too broad, and the retrieval results would not be precise enough. Therefore, a long document needs to be split into multiple relatively complete small segments, known as Chunks.

2. Query Phase: Finding Information and Answering Questions

The query process is executed each time a user asks a question:

User Question
   ↓
Embedding to Generate Query Vector
   ↓
Vector Database Performs Similarity Search
   ↓
Return Top-K Relevant Chunks
   ↓
Construct Prompt Context
   ↓
Large Language Model Generates Answer

Indexing and querying must use the same Embedding model. Vectors produced by different models may have different dimensions, and even if the dimensions are the same, the semantic spaces they reside in may not be consistent, making direct comparison impossible.

4. What Knowledge is Needed to Develop RAG?

RAG is not a single framework or model, but a data processing pipeline composed of multiple components. This column will focus on learning the following content.

1. LangChain: Stringing Together the RAG Process

LangChain is not a large language model, but a development framework for connecting documents, models, vector databases, and retrievers.

In this project, it is primarily responsible for providing unified abstractions:

Document: Represents a knowledge base document and its metadata;
TextSplitter: Splits long documents into multiple chunks;
Embeddings: Converts text into vectors;
VectorStore: Stores vectors and performs similarity retrieval;
Retriever: Queries relevant documents using a unified interface.

The key to mastering LangChain is not memorizing all APIs, but understanding the position of each object in the data flow.

2. Ollama: Running Models Locally

Ollama is used for downloading, managing, and running models, and provides model capabilities to applications via an HTTP API.

This project currently uses Ollama to run the Embedding model:

TypeScript Application
      ↓
OllamaEmbeddings
      ↓ HTTP
Ollama Service
      ↓
nomic-embed-text
      ↓
Text Vectors

Four concepts need to be distinguished here:

Ollama: The model running service;
nomic-embed-text: The model that actually performs vector calculations;
OllamaEmbeddings: LangChain's wrapper for the Ollama Embedding API;
LanceDB: The database for storing and searching vectors.

Ollama itself is not a specific model, and the Embedding model is not responsible for generating natural language answers.

3. Embedding Model: Enabling Semantic Computation on Text

The Embedding model converts text into a set of numbers:

"How does the system perform user authentication?"
           ↓
[0.12, -0.37, 0.88, ..., 0.24]

Texts with similar meanings typically have vectors that are closer together. Vector databases leverage this property for semantic retrieval.

This project defaults to using:

nomic-embed-text

Key points to understand later include:

The difference between embedDocuments() and embedQuery();
Model dimensions and semantic space;
Why the index needs to be rebuilt after changing the model;
How to choose a model for Chinese and mixed Chinese-English documents;
How to evaluate the actual retrieval effectiveness of a model.

4. Vector Store: Saving and Retrieving Vectors

The vector database is responsible for storing the following information:

The body text of the document chunk;
The vector generated by the Embedding model;
Metadata such as source, system, and document type.

When a user asks a question, the vector database compares the distance between the query vector and the document vectors, and returns the most relevant chunks.

This project chooses LanceDB because it can run directly in a local directory, making it suitable for personal projects and local knowledge base experiments without needing to deploy a separate database service first.

Key points to master include:

Similarity search;
Top-K;
Metadata filtering;
Table structure and data isolation;
Index rebuilding and incremental updates;
Relevance evaluation of retrieval results.

5. MCP: Providing Knowledge Base Capabilities to AI Clients

MCP stands for Model Context Protocol. It is used to provide tools, resources, and prompt capabilities to MCP-supporting AI clients in a standardized way.

Strictly speaking, MCP is not a mandatory component for implementing RAG retrieval. Even without MCP, we could call the knowledge base via an HTTP API, command line, or ordinary function calls.

In CorpRAG, MCP's role is the "external interface layer":

AI Client
   ↓ MCP Tool Call
CorpRAG
   ↓
Query LanceDB
   ↓
Return relevant knowledge chunks and sources
   ↓
The large language model in the AI client organizes the final answer

This approach decouples "knowledge retrieval" from "answer generation." CorpRAG focuses on maintaining and retrieving private knowledge, while the MCP-supporting host client is responsible for understanding user intent and generating the final answer.

Therefore, this column will not only introduce the traditional "retrieve-then-directly-call-LLM" process but will also practice the approach of "wrapping the RAG retriever as an MCP tool."

5. The Hands-on Project for This Column: CorpRAG

This column will use the current TypeScript project CorpRAG as the main thread, rather than using scattered demos detached from real-world scenarios.

The core technology stack currently adopted by the project is as follows:

Component	Choice	Role
Development Language	TypeScript	Writing the knowledge base service
RAG Framework	LangChain.js	Unifying document, chunking, Embedding, and retrieval processes
Model Service	Ollama	Running the Embedding model locally
Embedding Model	`nomic-embed-text`	Converting documents and questions into vectors
Vector Database	LanceDB	Storing vectors locally and performing similarity search
External Protocol	MCP	Exposing knowledge base query tools to AI clients
Document Format	Markdown	Storing technical and business decisions

1. How is Knowledge Organized?

The project distinguishes knowledge by business system, for example:

EXT
FR
GA

Each system contains two types of documents:

BADR: Business Decision Records;
FADR: Technical Decision Records.

After loading, documents are converted into LangChain Document objects and retain the following metadata:

{
  system: "ext",
  docType: "FADR",
  source: "data/EXT-FADR.md"
}

This metadata can be used for knowledge isolation, result filtering, and answer traceability.

2. Current Indexing Pipeline

The indexing process in the project can be summarized as:

System Markdown Documents
   ↓ document-loader.ts
LangChain Document[]
   ↓ text-splitter.ts
Document Chunks[]
   ↓ embeddings.ts
nomic-embed-text generates vectors
   ↓ vector-store.ts
Write to LanceDB by system and document type

The current default configuration is:

embeddingModel: "nomic-embed-text"
chunkSize: 500
chunkOverlap: 50
topK: 3

These values are not universal standard answers but a starting point for hands-on practice. Later, retrieval results will be verified with real questions, and then the chunk size, overlap length, and recall quantity will be adjusted.

3. Current Query Pipeline

When querying, the project opens the corresponding vector table based on the system and optional document type:

User Question
   ↓
ext_query / fr_query / ga_query
   ↓
OllamaEmbeddings generates query vector
   ↓
LanceDB similarity search
   ↓
Return body text, source, and document type

If BADR or FADR is not specified, the retrieval layer queries both types of knowledge separately and then merges the results, giving both business and technical decisions a chance to be recalled.

4. Why Provide Capabilities Externally via MCP?

The project will register MCP tools similar to the following:

ext_query
fr_query
ga_query
rebuild_kb

This way, an MCP-supporting AI client can actively call the knowledge base of a specific system based on the user's question, without needing to stuff all private documents into the conversation context at once.

It brings several direct benefits:

Retrieves only when needed;
Returns only the most relevant chunks;
Controls the context length sent to the model;
Preserves document sources;
The knowledge base can be re-indexed after updates without modifying the client's prompt.

6. What Does This Column Plan to Write?

The entire column will be divided into two phases: "Fundamental Knowledge Learning" and "Project Hands-on Practice." We will first understand the role and basic usage of each component in the RAG pipeline, then combine them to gradually complete the CorpRAG private knowledge base.

1. Fundamental Knowledge Learning

The initial plan is as follows:

Basic Introduction and Usage of MCP: Understand what problems MCP solves and how to provide tools to AI clients.
Basic Introduction and Usage of Ollama: Learn about downloading, managing, running local models, and API calls.
Basic Introduction and Usage of Vector Store: Understand how a vector database stores vectors and performs similarity retrieval.
Basic Introduction and Usage of Embedding Models: Understand how text is converted into vectors, and how to select and call vector models.
Basic Introduction and Usage of LangChain: Get to know core abstractions like Document, text chunking, Embedding, VectorStore, and Retriever.

2. CorpRAG Project Hands-on Practice

After understanding the above fundamentals, we will enter the complete project practice:

Continuously record project knowledge using Markdown;
Organize the RAG data flow with LangChain;
Generate vectors using Ollama and nomic-embed-text;
Store and retrieve knowledge using LanceDB;
Provide retrieval capabilities to AI clients via MCP;
Ensure every conclusion traces back to the original material as much as possible.

The hands-on part will unfold step-by-step around document loading, text chunking, vector generation, index building, semantic retrieval, and MCP tool wrapping, ultimately forming a searchable, traceable, and sustainably maintainable private knowledge base.

7. Before Starting, Set the Right Expectations

RAG is not a case of "throw documents into a vector database and the system will automatically become smart." The final result depends on the entire pipeline:

Document Quality
  × Chunk Quality
  × Embedding Model
  × Retrieval Strategy
  × Prompt Constraints
  × Large Language Model Capability
  = Final Answer Quality

When the correct information is not retrieved, the problem could be in the documents, chunking, model, or query. When the correct information is retrieved but the answer is wrong, the problem could be in the prompt or the generation model.

Therefore, the most important capability for developing RAG is not knowing how to call a specific high-level API, but being able to understand the complete process of data entering the system from documents and returning from the system as an answer.

This is also the reason why this column insists on focusing on project hands-on practice.

Summary

What this column aims to complete is not a general-purpose search engine, nor an all-knowing chatbot, but a knowledge infrastructure serving individuals and projects:

Continuously record project knowledge using Markdown;
Organize the RAG data flow with LangChain;
Generate vectors using Ollama and nomic-embed-text;
Store and retrieve knowledge using LanceDB;
Provide retrieval capabilities to AI clients via MCP;
Ensure every conclusion traces back to the original material as much as possible.

Finally, remember RAG in one sentence:

First, find trustworthy information from your own knowledge base, then let the large language model answer based on that information.