LangChain4j Is the Java-Native AI Framework That Spring AI Users Eventually Grow Into

Foreword

Recently, many friends have been asking me—"For developing AI applications in legacy Java projects, what framework should I use?"

My answer is: LangChain4j.

In this article today, I will break down LangChain4j's core concepts, underlying principles, and practical code from scratch.

I hope it will be helpful to you.

More project practices are available on the Java Commando website: susan.net.cn/project

1. What Exactly Is LangChain4j?

Some friends might say: "Isn't LangChain4j just the Java version of Python's LangChain?"

Not really.

LangChain4j does have a name related to Python's LangChain, but it is not a simple port of LangChain.

It was completely redesigned from scratch, following Java programming conventions—type safety, POJOs, annotations, interfaces, dependency injection, and fluent APIs.

As of 2026, LangChain4j has accumulated over 12,200 Stars and 2,300 Forks on GitHub, with the latest version being 1.15.1, maintaining an active development pace.

It natively supports 20+ LLM providers and 30+ vector stores, and has first-class integration with mainstream Java frameworks like Spring Boot, Quarkus, Helidon, and Micronaut.

In one sentence: LangChain4j is a large language model application development framework designed specifically for Java/Kotlin developers. It provides a unified, standardized API that abstracts away the underlying differences between various large models, vector databases, and document parsers, allowing Java developers to quickly build stable, scalable AI business applications without reinventing the wheel.

1.1 What Do You Face Without LangChain4j?

When directly interfacing with large model APIs, you have to deal with quite a few troublesome issues:

• Each vendor's API format, parameter names, and return structures are different;
• Every call requires manual handling of HTTP requests, JSON parsing, authentication, and retries;
• Multi-turn conversations require manual management of message history;
• Getting AI to answer based on your documents requires building RAG;
• Getting AI to check the weather or query orders requires tool calling.

LangChain4j's solution: These complex features are all packaged into ready-to-use components.

2. Understanding LangChain4j's Architecture at a Glance

Before writing code, let's establish an overall understanding.

LangChain4j's overall architecture is clearly layered, with five core modules supporting all AI business capabilities:

• Model Layer uniformly encapsulates the invocation logic for various large models and embedding models, shielding API differences;

• Memory Layer manages multi-turn conversation memory, supporting in-memory, persistent, and segmented memory;

• Document Layer supports loading, parsing, text splitting, and cleaning of multi-format documents like PDF, Word, and TXT;

• Embedding & Store Layer uniformly encapsulates vectorization and vector retrieval logic.

LangChain4j adopts a clear layered architecture design. The core abstraction layer (langchain4j-core) is the cornerstone of the entire framework, defining all core interfaces and data models.

3. LangChain4j's "Seven-Piece Set"

LangChain4j's component system is very clear. Below, I will break them down one by one.

3.1 Model

It is the "brain" of the AI.

Model is the entry point for interacting with large models. LangChain4j provides a unified interface to connect with different model providers.

There are currently two main types of APIs:

• LanguageModel: Both input and output are Strings, now used less and less.
• ChatModel: The most widely used API, receiving multiple ChatMessage objects as input and outputting an AiMessage, supporting multimodal inputs like text and images.

Example: Creating a ChatModel

// Taking OpenAI as an example
ChatModel model = OpenAiChatModel.builder()
    .apiKey(System.getenv("OPENAI_API_KEY"))
    .modelName("gpt-4")
    .build();

// Send a message
ChatResponse response = model.chat(
    UserMessage.from("Hello, please introduce yourself")
);
System.out.println(response.aiMessage().text());

3.2 ChatMessage

It is the basic unit of conversation.

LangChain4j supports five message types:

Message Type	Description	Main Purpose
UserMessage	User input message	User questions
AiMessage	AI-generated reply	Model output
SystemMessage	System message	Set the AI's role and behavior
ToolExecutionResultMessage	Tool execution result	Return result after function call
CustomMessage	Custom message	Extended scenarios

3.3 ChatMemory

It allows the AI to "remember" conversations.

Large models themselves are stateless and do not record conversation history. LangChain4j provides ChatMemory to manage conversation context.

Two built-in memory eviction strategies:

• MessageWindowChatMemory: Based on a message sliding window, only retaining the most recent N messages.
• TokenWindowChatMemory: Based on a token sliding window, only retaining the most recent N tokens.

// Create memory, retaining the last 10 messages
ChatMemory memory = MessageWindowChatMemory.builder()
    .maxMessages(10)
    .build();

// Add user message
memory.add(UserMessage.from("My name is Zhang San"));
// Get AI reply
AiMessage response = model.chat(memory.messages()).aiMessage();
memory.add(response);

// The next round of conversation will automatically carry history
memory.add(UserMessage.from("What is my name?"));
AiMessage response2 = model.chat(memory.messages()).aiMessage();
// The model will remember your name is Zhang San

💡 A Key Concept: LangChain4j provides "memory" rather than "history records."

Memory transforms history based on algorithms—evicting certain messages, summarizing multiple messages, removing unimportant details, injecting additional information, etc.

3.4 Tools

It gives AI "hands and feet."

Tools (function calling) is one of LangChain4j's most powerful features. It allows LLMs to call external tools—web search, calling external APIs, executing specific code, etc.

Example: Defining a math tool

import dev.langchain4j.agent.tool.Tool;

public class CalculatorTools {
    
    @Tool("Sums two given numbers")
    double sum(double a, double b) {
        return a + b;
    }
    
    @Tool("Returns the square root of a given number")
    double squareRoot(double x) {
        return Math.sqrt(x);
    }
}

⚠️ Key Point: Tool descriptions must be written clearly. Whether the AI can correctly call the tool depends entirely on this description!

Letting AI use tools

ChatModel model = OpenAiChatModel.builder()
    .apiKey(System.getenv("OPENAI_API_KEY"))
    .modelName("gpt-4")
    .build();

// Pass tools to the model
ChatRequest request = ChatRequest.builder()
    .messages(UserMessage.from("What is the square root of 475695037565?"))
    .toolSpecifications(ToolSpecifications.from(CalculatorTools.class))
    .build();

ChatResponse response = model.chat(request);
// AI will return a toolExecutionRequest, indicating it wants to call the squareRoot tool

Complete Tool Calling Flow: AiServices sends the message and tool schema to the LLM, the LLM replies with a function call (e.g., add(42, 58)), LangChain4j executes the Calculator method, and feeds the result back.

3.5 AiServices

It enables declarative AI development.

AiServices is LangChain4j's high-level API and the part that Java developers find most familiar.

Its core idea is interface-oriented programming: you just need to define a Java interface, use annotations to specify what capabilities it needs (system prompts, user message templates, memory, tools, etc.), and AiServices will generate a dynamic proxy object for you, automatically orchestrating all components internally.

Simplest AiService Example:

interface Assistant {
    String chat(String userMessage);
}

// Create AI service
Assistant assistant = AiServices.builder(Assistant.class)
    .chatLanguageModel(model)
    .build();

// Direct call
String reply = assistant.chat("Hello, please introduce yourself");
System.out.println(reply);

AiService with System Prompt and Memory:

interface ChatAssistant {
    
    @SystemMessage("You are a professional Java technical consultant, please answer questions in Chinese")
    String chat(@UserMessage String userMessage);
}

// Create AI service with memory
ChatMemory chatMemory = MessageWindowChatMemory.builder()
    .maxMessages(10)
    .build();

ChatAssistant assistant = AiServices.builder(ChatAssistant.class)
    .chatLanguageModel(model)
    .chatMemory(chatMemory)
    .build();

// Multi-turn conversation automatically carries memory
String reply1 = assistant.chat("My name is Zhang San");
String reply2 = assistant.chat("What is my name?"); // AI remembers your name is Zhang San

AiServices supports capabilities including:

• Static/Dynamic System Messages: Configured via @SystemMessage annotation or systemMessageProvider().
• Static/Dynamic User Messages: Via @UserMessage annotation or @UserMessage annotated parameters.
• Shared Memory: Configured via chatMemory(ChatMemory).
• Multi-user Memory: Via chatMemoryProvider() and @MemoryId annotated parameters.
• RAG Retrieval Augmentation: Configured via contentRetriever() or retrievalAugmentor().

3.6 RAG

It allows AI to have "evidence-based answers."

RAG is one of LangChain4j's core capabilities.

Its flow is: User asks a question → Retrieve relevant documents from a knowledge base → Send the question and retrieved documents to the AI together → AI generates a reply based on the documents.

In LangChain4j, the core component of RAG is RetrievalAugmentor.

It acts like the "central processor" of the RAG system, specifically responsible for "enriching" the user's question—by calling various retrieval channels, it "attaches" found relevant knowledge snippets to the original question, allowing the large model to reference these materials when answering.

// 1. Load document
Document document = FileSystemDocumentLoader.loadDocument("knowledge.txt");

// 2. Split
DocumentSplitter splitter = DocumentSplitters.recursive(300, 0);
List<TextSegment> segments = splitter.split(document);

// 3. Vectorize and store
EmbeddingModel embeddingModel = new BgeSmallEnV15EmbeddingModel();
EmbeddingStore<TextSegment> embeddingStore = new InMemoryEmbeddingStore<>();
for (TextSegment segment : segments) {
    Embedding embedding = embeddingModel.embed(segment).content();
    embeddingStore.add(embedding, segment);
}

// 4. Create ContentRetriever
ContentRetriever retriever = EmbeddingStoreContentRetriever.builder()
    .embeddingStore(embeddingStore)
    .embeddingModel(embeddingModel)
    .maxResults(3)
    .build();

// 5. Create AiService with RAG
Assistant assistant = AiServices.builder(Assistant.class)
    .chatLanguageModel(model)
    .contentRetriever(retriever)
    .build();

// 6. Ask a question, AI will answer based on the knowledge base
String answer = assistant.chat("What is the company's leave request process?");

Standard RAG can be further customized: Load Markdown documents and split on demand, supplement file name information, customize Embedding models, customize content retrievers.

Advanced RAG supports features like query transformers, query routing, content aggregators, and content injectors, pipelining the entire RAG process (RAG Pipeline).

3.7 MCP Protocol

It gives AI a "USB interface."

Some friends might ask: "Besides custom tools, can LangChain4j connect to external services?"

MCP (Model Context Protocol) is exactly for this.

You can think of MCP as the "USB interface" for AI applications. It provides a standardized way for AI to interact with external tools, resources, and services.

Integrating MCP in LangChain4j is very convenient:

<!-- Introduce MCP dependency -->
<dependency>
    <groupId>dev.langchain4j</groupId>
    <artifactId>langchain4j-mcp</artifactId>
    <version>1.1.0-beta7</version>
</dependency>

@Configuration
public class McpConfig {
    @Bean
    public McpToolProvider mcpToolProvider() {
        // 1. Configure communication method with MCP service (SSE)
        McpTransport transport = new HttpMcpTransport.Builder()
            .sseUrl("https://open.bigmodel.cn/api/mcp/web_search/sse?Authorization=" + apiKey)
            .build();
        // 2. Create MCP client
        McpClient mcpClient = new DefaultMcpClient.Builder()
            .transport(transport)
            .build();
        // 3. Get tool provider from MCP client
        return McpToolProvider.builder()
            .mcpClients(mcpClient)
            .build();
    }
}

4. Hands-on Practice

Talk is cheap.

Below, I will use Spring Boot + LangChain4j to quickly build an AI chat application.

4.1 Step 1: Create Project and Add Dependencies

<properties>
    <java.version>21</java.version>
    <spring-boot.version>3.4.5</spring-boot.version>
    <langchain4j.version>1.15.1</langchain4j.version>
</properties>

<dependencies>
    <!-- Spring Boot Web -->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>
    
    <!-- LangChain4j Core -->
    <dependency>
        <groupId>dev.langchain4j</groupId>
        <artifactId>langchain4j</artifactId>
        <version>${langchain4j.version}</version>
    </dependency>
    
    <!-- OpenAI Compatible Adapter (compatible with DeepSeek/Ollama/DashScope, etc.) -->
    <dependency>
        <groupId>dev.langchain4j</groupId>
        <artifactId>langchain4j-open-ai</artifactId>
        <version>${langchain4j.version}</version>
    </dependency>
    
    <!-- Spring Boot Integration -->
    <dependency>
        <groupId>dev.langchain4j</groupId>
        <artifactId>langchain4j-spring-boot-starter</artifactId>
        <version>${langchain4j.version}</version>
    </dependency>
</dependencies>

Key Understanding: langchain4j-open-ai is not just for interfacing with OpenAI; it is an OpenAI-compatible protocol adapter. Any service providing a /v1/chat/completions endpoint (DeepSeek, Ollama, SiliconFlow, Tongyi Qianwen DashScope) can be used.

4.2 Step 2: Configure application.yml

langchain4j:
  open-ai:
    chat-model:
      api-key: ${OPENAI_API_KEY}
      model-name: gpt-4
      temperature: 0.7
      log-requests: true
      log-responses: true
    embedding-model:
      api-key: ${OPENAI_API_KEY}
      model-name: text-embedding-ada-002

4.3 Step 3: Define AiService Interface

package com.example.service;

import dev.langchain4j.service.SystemMessage;
import dev.langchain4j.service.UserMessage;
import dev.langchain4j.service.MemoryId;
import dev.langchain4j.service.spring.AiService;

@AiService
public interface ChatAssistant {
    
    @SystemMessage("You are a professional AI assistant, please answer questions in Chinese, concisely and friendly.")
    String chat(@UserMessage String userMessage);
    
    // Multi-user memory with session ID
    @SystemMessage("You are a professional AI assistant, please answer questions in Chinese.")
    String chat(@MemoryId String sessionId, @UserMessage String userMessage);
}

4.4 Step 4: Write Controller

package com.example.controller;

import com.example.service.ChatAssistant;
import org.springframework.web.bind.annotation.*;

@RestController
@RequestMapping("/api/chat")
public class ChatController {
    
    private final ChatAssistant chatAssistant;
    
    public ChatController(ChatAssistant chatAssistant) {
        this.chatAssistant = chatAssistant;
    }
    
    @PostMapping
    public String chat(@RequestBody ChatRequest request) {
        return chatAssistant.chat(request.getMessage());
    }
    
    @PostMapping("/session")
    public String chatWithSession(@RequestBody SessionChatRequest request) {
        return chatAssistant.chat(request.getSessionId(), request.getMessage());
    }
}

record ChatRequest(String message) {}
record SessionChatRequest(String sessionId, String message) {}

4.5 Step 5: Start Application

@SpringBootApplication
public class Application {
    public static void main(String[] args) {
        runApplication(Application.class, args);
    }
}

After starting, access POST /api/chat to chat with the AI.

In less than 50 lines of code, a complete AI chat service is up and running.

5. Advanced Usage

5.1 Structured Output

It allows AI to return Java objects.

Many LLMs support generating structured format (usually JSON) output, which can be easily mapped to Java objects and used in applications.

// 1. Define the data structure to extract
public class PersonInfo {
    public String name;
    public int age;
    public String city;
}

// 2. Specify return type in AiService
interface PersonExtractor {
    @UserMessage("Extract person information from the following text: {{text}}")
    PersonInfo extractPerson(@V("text") String text);
}

// 3. Call
PersonExtractor extractor = AiServices.builder(PersonExtractor.class)
    .chatLanguageModel(model)
    .build();

PersonInfo info = extractor.extractPerson("Zhang San, 28 years old this year, lives in Beijing");
System.out.println(info.name); // Zhang San
System.out.println(info.age); // 28

5.2 Streaming Response

It can output word by word like ChatGPT.

Streaming transmission is implemented through StreamingChatLanguageModel, without waiting for the complete answer to load, responding to the user in real-time.

StreamingChatLanguageModel model = OpenAiStreamingChatModel.builder()
    .apiKey(System.getenv("OPENAI_API_KEY"))
    .modelName("gpt-4")
    .build();

model.chat(UserMessage.from("Write a poem about Java"), 
    new StreamingResponseHandler<AiMessage>() {
        @Override
        public void onNext(String token) {
            System.out.print(token); // Print each token in real-time
        }
        @Override
        public void onComplete(Response<AiMessage> response) {
            System.out.println("\n--- Generation Complete ---");
        }
        @Override
        public void onError(Throwable error) {
            error.printStackTrace();
        }
    }
);

6. Pros and Cons

Pros

1. Unified API, Seamless Multi-Model Switching LangChain4j provides a unified API that shields the differences between different LLM providers and vector stores. Switching from OpenAI to Tongyi Qianwen only requires changing the configuration, with almost no changes to business code.

2. Ultimate Multi-Model Adaptation Natively supports 15+ mainstream large models including OpenAI, Tongyi Qianwen, Wenxin Yiyan, Llama3, Claude, etc., with seamless switching using one set of code.

3. Declarative Development, Extremely Efficient AiServices allows developers to simply define interfaces and add annotations, and the framework automatically generates the implementation. Say goodbye to redundant boilerplate code.

4. Modular, Pluggable Architecture Conversation, memory, document loading, splitting, vector storage, and tool calling components are completely decoupled and can be combined as needed.

5. Full-Scenario Capability Coverage Natively supports RAG, streaming dialogue, multi-turn memory, function calling, Agent intelligent orchestration, and document parsing.

6. Perfect Integration with Spring Ecosystem Provides Spring Boot Starter, seamlessly integrating into the mainstream Java technology stack.

7. Active Community, Rapid Iteration Since its launch in early 2023, the community has remained active. In 2026, multiple versions including 1.14.0 and 1.15.1 have been released.

Cons

1. Steep Learning Curve Requires understanding new concepts in LLM application development: Prompt templates, memory management, tool calling, RAG, Agents, etc. Compared to Spring AI, LangChain4j has more configuration and a steeper learning curve, but the advantage is being able to grasp details and have full control.

2. Fast Version Iteration, Potential Breaking Changes Frequent version updates may lead to API changes, requiring attention to Release Notes when upgrading.

3. Incomplete Official Documentation Some developers reflect that "they simply cannot find official documentation for key content, and important content that should be there is not introduced at all."

4. Some Advanced Features Still Under Development Although core features are in place, some features are still under development.

7. LangChain4j vs Spring AI

Many developers struggle with the choice: Should I choose Spring AI or LangChain4j?

Comparison Dimension	Spring AI	LangChain4j
Core Positioning	AI infrastructure for Spring ecosystem	LLM application development toolkit on JVM
Framework Dependency	Strong dependency on Spring Boot	Not dependent on Spring, a general Java library
Feature Richness	Basic features	Richer features, more flexible
Learning Curve	Lower	Higher
Applicable Scenarios	Simple features, quick integration	Complex workflows, Agents, advanced customization

Selection Advice:

• If you are a deep user of the Spring ecosystem and just starting to learn AI, it is recommended to start with Spring AI for quick model integration.
• When you need to build complex Agents, RAG, or workflows, LangChain4j is recommended.
• Both can also be mixed—using LangChain4j's specific capabilities as needed within a Spring Boot project.

Essential Difference: If Spring AI is a skilled assembler, then LangChain4j is more like a logically rigorous architect.

8. Production Pitfall Avoidance Guide

Some friends might step into pitfalls during practice. Here, I've compiled a few common issues:

Pitfall 1: Tool call descriptions must be written clearly Whether the AI can correctly call a tool depends entirely on the @Tool description. If the description is too vague, the AI might not know when to call it.

Pitfall 2: Watch out for configuration conflicts when switching between multiple models When configuring multiple model providers simultaneously, you need to explicitly specify the provider for each named model.

Pitfall 3: Conversation memory is not history records LangChain4j provides "memory" rather than complete "history records." Memory transforms history based on algorithms.

Pitfall 4: Inconsistent model capabilities The capabilities of different models within the same brand vary greatly. Run a minimal viable Demo first to verify.

Pitfall 5: Cases where AiMessage.text() is null In multi-Agent setups, when the LLM returns a pure tool call response (no text content), processing the text field of AiMessage might throw an NPE.

Pitfall 6: Dependency versions must match LangChain4j's version needs to align with the backend model SDK version to avoid compatibility issues.

More project practices are available on the Java Commando website: susan.net.cn/project

9. Final Words

Back to the original question: For Java AI application development, what framework should you use?

If you are a Java backend developer looking to quickly integrate AI capabilities into a Spring Boot project—LangChain4j is currently one of the best choices.

It is not a simple port of Python's LangChain, but an AI application development framework designed from scratch for Java, following Java programming conventions.

It provides a unified API, declarative AiServices, a rich component library, and seamless integration with the Spring ecosystem.