跪拜 Guibai
← Back to the summary

LangChain.js Memory: From Session Coherence to Long-Term Recall

Giving AI a "Soul": The Ultimate Practical Guide to Short-Term and Long-Term Memory in LangChain.js

When building RAG (Retrieval-Augmented Generation) applications, we often focus too much on the accuracy of "retrieval" and neglect the coherence of "conversation." An AI without memory is like a patient with severe anterograde amnesia—it is knowledgeable but cannot remember what you said a second ago.

In the LangChain.js ecosystem, the Memory module is the bridge connecting the LLM with the user's historical interactions. It is not just simple text concatenation; it is an art of context window management, information compression, and persistent storage.

This article will take you from the underlying BaseChatMessageHistory to the upper-level RunnableWithMessageHistory, and then to complex long-term memory architectures, providing a comprehensive breakdown of how to build AI applications with a "soul" in a Node.js environment.

Chapter 1: The Cornerstone of Memory — The LangChain.js Message System

Before diving into Memory, we must first understand how LangChain.js defines "memory." Similar to the Python version, the core of the JS version lies in the Message object.

1.1 Types of Messages

In code, memory is essentially an array of BaseMessage objects. Understanding these types is crucial for subsequent processing:

1.2 Why Can't We Just Stuff All History into the LLM?

This is the most common mistake beginners make. An LLM's Context Window is limited (e.g., 4k, 8k, 32k tokens).

  1. Token Explosion: As the conversation progresses, historical messages quickly exhaust the token quota.
  2. Lost in the Middle: Research shows that LLMs pay less attention to information in the middle of long texts; excessive irrelevant history can interfere with the current answer.
  3. Cost and Latency: More tokens mean higher API fees and longer wait times.

Therefore, the core value of Memory is: retaining the most valuable information within a limited window.

Chapter 2: Short-Term Session Memory — Making Conversations "Coherent"

Short-term Memory typically refers to context management within the current session window. Its lifecycle is usually bound to a single session.

2.1 Core Component: ChatMessageHistory

ChatMessageHistory is the standard interface in LangChain.js for storing and retrieving message lists. It doesn't decide "how to compress" itself; it is only responsible for "storing" and "retrieving."

The simplest implementation is in-memory storage (for testing only):

import { InMemoryChatMessageHistory } from "@langchain/core/chat_history";

const history = new InMemoryChatMessageHistory();
await history.addUserMessage("Hello, my name is Xiao Ming");
await history.addAIChatMessage("Hello Xiao Ming! How can I help you?");

const messages = await history.getMessages();
console.log(messages); 
// [HumanMessage, AIMessage]

2.2 Production-Grade Wrapper: RunnableWithMessageHistory

In actual LangChain.js (v0.1+) development, we no longer manually call history.getMessages() and then concatenate them into the Prompt. The officially recommended approach is to use RunnableWithMessageHistory. This is a high-level wrapper that automatically handles the entire process of "reading history -> concatenating into input -> calling LLM -> saving new messages."

Scenario: Building a Stateful Customer Service Bot

Suppose we need to maintain independent conversation histories for each session_id.

import { ChatOpenAI } from "@langchain/openai";
import { ChatPromptTemplate, MessagesPlaceholder } from "@langchain/core/prompts";
import { RunnableWithMessageHistory } from "@langchain/core/runnables";
import { UpstashRedisChatMessageHistory } from "@langchain/community/stores/message/upstash_redis";

// 1. Define the Prompt, note that MessagesPlaceholder must be included here
// It is a placeholder telling the LLM "historical messages will be inserted here"
const prompt = ChatPromptTemplate.fromMessages([
  ["system", "You are a helpful customer service assistant."],
  new MessagesPlaceholder("history"), // Key placeholder
  ["human", "{input}"],
]);

// 2. Define the LLM
const model = new ChatOpenAI({ temperature: 0.7 });

// 3. Assemble the chain
const chain = prompt.pipe(model);

// 4. Wrap into RunnableWithMessageHistory
const chainWithHistory = new RunnableWithMessageHistory({
  runnable: chain,
  
  // Factory function for getting history records
  // Returns the corresponding History instance based on sessionId each time it's called
  getMessageHistory: (sessionId) => 
    new UpstashRedisChatMessageHistory({
      sessionId,
      config: {
        url: process.env.UPSTASH_REDIS_REST_URL!,
        token: process.env.UPSTASH_REDIS_REST_TOKEN!,
      },
    }),
    
  // Specify input and output fields
  inputMessagesKey: "input",
  historyMessagesKey: "history",
});

// 5. Invoke
const response = await chainWithHistory.invoke(
  { input: "What was the order number I just mentioned?" },
  { configurable: { sessionId: "user_001_session_A" } }
);

Tips:

2.3 Advanced Technique: Sliding Window and Truncation

The default InMemoryChatMessageHistory or Redis implementation usually returns all history. When the conversation is long, this will blow up the token limit. We need to introduce a buffer window.

Although LangChain.js currently doesn't have as rich a set of built-in BufferWindowMemory classes as Python, we can implement this through custom getMessageHistory logic:

// Custom logic: Only take the last N messages
const getLastNMessages = async (historyInstance, n = 10) => {
  const allMessages = await historyInstance.getMessages();
  // Simple slice operation, keeping only the last N messages
  return allMessages.slice(-n); 
};

// Apply in RunnableWithMessageHistory
// Note: This requires slightly modifying the RunnableWithMessageHistory configuration or manually implementing a Wrapper
// Currently, the JS version recommends handling truncation directly at the Prompt level or in a custom Runnable

Better Solution: Use a Token counter for dynamic truncation. Before passing messages to the LLM, calculate the token count. If it exceeds a threshold (e.g., 3000 tokens), start dropping messages from the earliest until the limit is met. This is safer than a fixed number of messages.

Chapter 3: Long-Term Session Memory — Building an AI That "Understands You"

Short-term memory solves "what was just said," while long-term memory solves "who you are" and "what our previous consensus was." In RAG systems, this is usually implemented through a Vector Store.

3.1 Core Concept: Entity Extraction and Knowledge Graphs

Long-term memory is not about saving every single sentence, but about distillation. The process is usually:

  1. Observation: Capture the current conversation content.
  2. Extraction: Use an LLM to extract key facts from the conversation (such as user preferences, names, important events).
  3. Storage: Vectorize these facts and store them in a database.
  4. Retrieval: At the start of a new conversation, retrieve relevant long-term memories based on the current question.

3.2 Practical: Long-Term Memory Based on VectorStore

We can utilize LangChain.js's VectorStoreRetrieverMemory pattern.

Step 1: Initialize Vector Storage

Here we use ChromaDB as an example (Pinecone, Milvus, etc., can also be used).

import { Chroma } from "@langchain/community/vectorstores/chroma";
import { OpenAIEmbeddings } from "@langchain/openai";

const vectorStore = await Chroma.fromExistingCollection(
  new OpenAIEmbeddings(),
  { collectionName: "long_term_memory" }
);

Step 2: Build the Memory Retrieval Chain

We need to create a mechanism that, before each conversation, first searches the vector store for any old news about this user.

import { createStuffDocumentsChain } from "langchain/chains/combine_documents";
import { ChatPromptTemplate } from "@langchain/core/prompts";

// Define a Prompt specifically for injecting background knowledge
const memoryPrompt = ChatPromptTemplate.fromTemplate(
  `Below is long-term memory information about the user. If relevant to the current question, please refer to it in your answer:
  <context>
  {context}
  </context>
  
  Current user question: {input}`
);

// Create a retriever
const retriever = vectorStore.asRetriever({ k: 3 }); // Only take the top 3 most relevant

// Create a document processing chain (merges retrieved memory fragments into a string)
const combineDocsChain = await createStuffDocumentsChain({
  llm: new ChatOpenAI(),
  prompt: memoryPrompt,
});

// Create the retrieval chain
const retrievalChain = await createRetrievalChain({
  combineDocsChain,
  retriever,
});

Step 3: Write to Long-Term Memory (Background Task)

You cannot do this synchronously while the user is waiting for a reply; it's too slow. The usual practice is to execute it asynchronously.

import { Document } from "@langchain/core/documents";

async function saveToLongTermMemory(sessionId, userMessage, aiResponse) {
  // 1. Use LLM to extract key information
  const extractionPrompt = `Extract key facts about the user (preferences, personal information, important events) from the following conversation.
  If there are no important facts, return empty. Format as a JSON array.
  User: ${userMessage}
  AI: ${aiResponse}`;
  
  const extractionResult = await llm.invoke(extractionPrompt);
  // Assume facts = ["User likes lattes", "User lives in Shanghai"] are parsed
  
  // 2. Store in vector database
  if (facts.length > 0) {
    const docs = facts.map(fact => new Document({
      pageContent: fact,
      metadata: { sessionId, type: "user_preference", timestamp: Date.now() }
    }));
    await vectorStore.addDocuments(docs);
  }
}

3.3 Advanced Technique: Zep and Mem0

Writing your own extraction logic is cumbersome. In the JS ecosystem, it is highly recommended to use dedicated memory management services, such as Zep or Mem0.

Taking Zep as an example (it has an excellent JS SDK):

import { ZepClient } from "@getzep/zep-js";

const client = new ZepClient({ apiKey: "YOUR_API_KEY" });

// Add messages (automatically handles summarization, entity extraction, vectorization)
await client.memory.add(sessionId, {
  messages: [{ role: "human", content: "I like science fiction novels" }]
});

// Get memory (automatic summarization + vector retrieval)
const memory = await client.memory.get(sessionId, { 
  lastn: 5, // Last 5 rounds
  searchType: "similarity" // Enable semantic search
});

This approach turns "long-term memory" into a black-box service, greatly reducing development complexity.

Chapter 4: Memory Fusion Strategies in RAG

In RAG systems, we face dual retrieval: Knowledge Base Retrieval (external data) + Memory Retrieval (internal data). How to fuse them is key.

4.1 Strategy 1: Parallel Retrieval

Send the "user question" to both the "Knowledge Base Retriever" and the "Memory Retriever" simultaneously.

// Pseudocode structure
const finalChain = RunnableBranch.from([
  // Branch logic...
]).pipe({
  context: RunnableMap.from({
    knowledge: knowledgeRetriever, // Search documents
    memory: memoryRetriever        // Search history
  }).pipe((inputs) => formatContext(inputs.knowledge, inputs.memory)),
  question: (input) => input.question
}).pipe(llm);

4.2 Strategy 2: Memory as Metadata Filtering

If the user's long-term memory contains "I only care about news in the financial sector," then when retrieving from the external knowledge base, this preference should be passed as a Metadata Filter to the vector database, thereby narrowing the search scope.

// Assume filter criteria are extracted from memory
const memoryFilter = { category: "finance" };

const results = await vectorStore.similaritySearch(query, 5, memoryFilter);

4.3 Strategy 3: HyDE (Hypothetical Document Embeddings) Combined with Memory

Sometimes the user's question is very vague (e.g., "How about that one?"). Direct retrieval works poorly in this case.

  1. First, check short-term memory to confirm what "that one" refers to.
  2. Use the LLM to generate a hypothetical complete question.
  3. Use the generated complete question to retrieve from the RAG knowledge base.

Chapter 5: Pitfall Avoidance Guide and Performance Optimization

5.1 Avoid "Ghost Context"

When using RunnableWithMessageHistory, ensure that the variable name of MessagesPlaceholder (e.g., history) strictly matches historyMessagesKey. Otherwise, the LLM will receive an empty context or throw an error.

5.2 Serialization Pitfalls

In Node.js, if you use Redis to store messages, be aware that objects like AIMessage contain complex methods. Do not directly JSON.stringify the entire object. Correct Approach: Use the utility functions mapChatMessagesToStoredMessages and mapStoredMessagesToChatMessages provided by LangChain for conversion.

import { mapChatMessagesToStoredMessages } from "@langchain/core/messages";

const stored = await mapChatMessagesToStoredMessages(messages);
await redis.set(key, JSON.stringify(stored));

5.3 Privacy and Security

Long-term memory is a disaster zone for privacy leaks.

5.4 Debugging Tips

The memory system is a black box and difficult to debug. Suggestion: During the development phase, use LangSmith or simple Console Log to print the complete Prompt finally sent to the LLM.

// Debugging middleware
const debuggableChain = chain.pipe((output) => {
  console.log("=== Final Prompt Sent to LLM ===");
  // You need to intercept the Input to see it, usually done in a RunnableMap
  return output;
});

After seeing the complete Prompt, you can judge whether "the memory wasn't retrieved" or "there was too much memory, confusing the LLM."

Chapter 6: Future Outlook — Adaptive Memory

Most current memory is passive. The future direction is active memory. AI should be able to realize: "The information I currently have is insufficient to answer this question; I need to ask the user to update my long-term memory."

For example:

User: "Book me a flight." AI (Internal Thought): Memory only has that he prefers window seats, but the destination is unknown. AI (Response): "Okay, shall I book you a window seat as usual? Also, where are you planning to go this time?"

This capability requires combining the Agent framework, treating Memory as a Tool for the AI to call autonomously (read_memory, write_memory), rather than just as a backdrop for the Prompt.

Conclusion

Building a RAG memory system in the JavaScript ecosystem is both challenging and fun. From the simple start of InMemoryChatMessageHistory, to the engineering encapsulation of RunnableWithMessageHistory, to the long-term memory network combined with Vector Stores, every step is to make AI more human-like.

Remember, Memory is not just a technical implementation; it is product design. You need to decide based on the business scenario:

I hope this guide can light a lamp on your LangChain.js development path. Now, open your IDE and give your AI a true "soul"!