A 3-Day AI Customer Service Agent with Spring AI 1.0 and DeepSeek

Honestly, I wasn't confident when I took on this task.

Last month, the product manager came to me and said the customer service team was overwhelmed—over 800 tickets a day, four agents working shifts couldn't keep up, and user satisfaction had dropped to 72%. The leadership's message was: you're the tech guy, try building an AI customer service agent.

My first reaction was: here we go, reinventing the wheel again. The company had tried an intelligent customer service system before, using a commercial solution that cost tens of thousands. The answers were completely off-topic, and it was taken offline after just two weeks. This time, I was pretty nervous about it.

But there was one difference this time—Spring AI 1.0 officially went GA in May. I'd been following this project for a while, experimenting since version 0.8. Back then, there were plenty of bugs and the API kept changing. But after the 1.0 GA release, I went through it again and found it was genuinely usable.

Here's the full record of the process, no sugar-coating, including all the pitfalls I ran into.

The Result First

It took 3 days to build an AI customer service agent that can automatically handle 60% of common questions with about 85% accuracy (much better than the previous commercial solution). The boss saw the demo and immediately approved a trial run.

The tech stack is simple: Spring Boot 4 + Spring AI 1.0 + DeepSeek API + Redis for vector caching.

Day 1: Getting the Minimum Viable Loop Running

The only goal for the first day was to get the AI to answer company business questions.

Why was the previous commercial solution so bad? Because it only had a single large model API with no context at all. It just threw questions like "how do I return an item" directly at the model. The large model doesn't know your company's return policy, so it could only make things up.

So the core idea is RAG (Retrieval-Augmented Generation): first, load the company knowledge base, then when a user asks a question, retrieve relevant documents first, and finally feed both the documents and the question to the large model.

Doing this with Spring AI 1.0 isn't complicated. First, add the dependencies:

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-starter-model-openai</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-starter-vector-store-redis</artifactId>
</dependency>

Then configure the DeepSeek API in application.yml (Spring AI's OpenAI-compatible interface can directly use DeepSeek):

spring:
  ai:
    openai:
      api-key: ${DEEPSEEK_API_KEY}
      base-url: https://api.deepseek.com
      chat:
        options:
          model: deepseek-chat
          temperature: 0.3  # Lower temperature for customer service scenarios for more stable answers

The temperature parameter is critical. At first, I used the default 0.7, and the same question got different answers each time. The customer service supervisor complained immediately. After adjusting it to 0.3, the answers became much more stable.

The next step was loading the knowledge base documents into the Redis vector store. The customer service team had a bunch of Word docs and PDFs—product FAQs, return and exchange policies, common troubleshooting guides, etc. Spring AI 1.0's built-in document reader can read PDFs directly:

@Service
public class KnowledgeImportService {

    private final VectorStore vectorStore;

    public void importKnowledgeBase(String filePath) {
        // Read PDF
        var reader = new PagePdfDocumentReader(filePath);
        var docs = reader.get();

        // Split into chunks, roughly 500 tokens each
        var splitter = new TokenTextSplitter(500, 200, 10, 5000, true);
        var chunks = splitter.apply(docs);

        // Write to Redis vector store
        vectorStore.add(chunks);
        
        log.info("Import complete, total {} document chunks", chunks.size());
    }
}

I hit a pitfall here. Initially, the chunks were too small (200 tokens), resulting in fragmented context. The retrieved chunks were incomplete, and the AI's answers were often out of context. After adjusting to 500 tokens, it was much better. There's no standard answer for this parameter; you have to experiment based on your document type.

By the end of the first day, the minimum viable loop was running—user inputs a question, the system retrieves from the knowledge base, calls the large model, and returns an answer. It was still rough, but at least it could answer questions like "What is your return policy?"

Day 2: Adding Function Calling to Get Real Work Done

Just answering questions isn't enough. A high-frequency need in customer service is checking order status.

When a user says "Where is my order?", RAG alone can't help—order data is in the database, not the knowledge base. This is where Spring AI's Function Calling comes in.

Simply put, it lets the AI call your Java methods. Define a function:

@Bean
@Description("Query order status and logistics information by order number")
public Function<OrderQuery, OrderInfo> queryOrder(OrderService orderService) {
    return query -> orderService.queryOrderInfo(query.orderNo());
}

public record OrderQuery(String orderNo) {}
public record OrderInfo(String status, String logistics, String estimatedArrival) {}

Then bind this function in the ChatClient:

var response = chatClient.prompt()
    .system("""
        You are the company's AI customer service assistant. When answering user questions:
        1. Base your answers on the knowledge base content, do not fabricate
        2. If the user provides an order number, call queryOrder to check the order status
        3. Be friendly but concise, avoid unnecessary chatter
        """)
    .user(userMessage)
    .functions("queryOrder")
    .call()
    .content();

The effect is this—when a user says "Help me check the status of order DD20260720001", the AI automatically recognizes the intent to check an order, extracts the order number, calls the queryOrder method, gets the result, and organizes it into a natural language reply.

This is much better than the previous commercial solution. That one required writing a bunch of intent recognition rules; now the large model understands the intent on its own.

But there's a pitfall here too. Function Calling has a limitation: the function's parameters and return value are serialized into JSON and passed to the large model. If your return value is too large (for example, returning a giant DTO with 100 fields), token consumption will skyrocket, and the model can easily get "dazzled" and miss the key points. So the return value must be streamlined, returning only the information the AI needs.

// Don't do this — return the entire Order entity
public record OrderInfo(Order order) {}  // Order has 50 fields...

// Do this instead — return only the needed fields
public record OrderInfo(String status, String logistics, String estimatedArrival) {}

Day 3: Adding Memory, Fallback, and Monitoring

The third day was mainly about polishing the details.

Conversation Memory. Users often ask several questions in a row, like "What's the return policy?" → "Who pays for the shipping?" → "How long until I get my refund?". The second and third questions can't be answered without context. Spring AI 1.0's memory feature:

@Bean
public ChatClient chatClient(ChatClient.Builder builder, ChatMemory memory) {
    return builder
        .defaultSystem("You are the company's AI customer service assistant...")
        .defaultAdvisors(new MessageChatMemoryAdvisor(memory))
        .build();
}

After adding this, the AI can remember the context within the same session. I used a Redis-backed ChatMemory implementation here because the default InMemoryChatMemory is lost on restart.

Fallback Mechanism. I think this is the most important part. AI isn't omnipotent; when it encounters a question it can't answer, it must not be allowed to make things up. I added a hard constraint in the system prompt:

If there is no relevant information in the knowledge base, you must reply: "I'm unable to answer this question at the moment. I'm transferring you to a human agent."
Never fabricate an answer.

Then in the code, detect this reply and automatically trigger the human handoff process:

if (response.contains("transferring you to a human agent")) {
    // Automatically create a ticket and transfer to a human agent
    ticketService.createEscalationTicket(userId, userMessage, conversationId);
}

After this fallback logic went live, feedback from the customer service team improved significantly—at least there were no more user complaints caused by the AI making up answers.

Monitoring. This is easy to overlook but very important. You need to know how many questions the AI answered, its accuracy rate, and the human handoff rate. Spring AI 1.0 integrates Micrometer, which can be enabled directly in application.yml:

management:
  endpoints:
    web:
      exposure:
        include: health,metrics,prometheus

Then I wrote a simple statistics dashboard to check these metrics daily:

Total AI replies / Human handoffs (handoff rate)
Average response time
User satisfaction (subsequent tagging by the customer service team)

Real Data After One Week Online

Metric	AI Agent	Before
Daily Tickets Handled	520	800 (all human)
Auto-Resolution Rate	62%	0% (all human)
Average Response Time	3 seconds	15 minutes
User Satisfaction	83%	72%
Human Handoff Rate	38%	—

The 62% auto-resolution rate is honestly higher than I expected. The remaining 38% handed off to humans are mainly complex complaints and scenarios requiring refund operations—these should be handled by people anyway.

Satisfaction went from 72% to 83%, an 11-point increase. The biggest improvement is response speed; previously, users waited 15 minutes for an answer to a question, now they get a reply in 3 seconds.

Pitfalls and Lessons Learned

Pitfall 1: Knowledge base quality determines everything. 80% of the AI's answer accuracy depends on the knowledge base. Initially, the documents customer service gave me were a mess—old policies from 2019 that hadn't been updated, and three different versions of the same question. After spending half a day cleaning up the documents, the AI's answer accuracy immediately jumped a level.

Pitfall 2: DeepSeek occasionally acts up. Once, a user asked a completely unrelated question (about the weather), and the AI seriously answered the weather question using the company's return policy format. Later, I added a constraint in the system prompt: "Only answer questions related to the company's business; politely decline unrelated questions."

Pitfall 3: API rate limiting when concurrency spikes. DeepSeek's API has concurrency limits, and occasionally returns 429 errors during peak times. I added a local queue for peak shaving, and it's not a big issue. If you're using a commercial API, remember to calculate the cost—we have 500+ calls a day, and the monthly fee is around 200-300 RMB, much cheaper than human customer service.

Some Thoughts on Technology Choices

Some might ask, why not use LangChain4j?

I compared them. LangChain4j is indeed good, with an active community and comprehensive features. But I chose Spring AI for two reasons:

First, seamless integration with Spring Boot. Dependency injection, configuration management, Actuator monitoring—all ready to use out of the box, with no extra learning cost. For a team like ours already on the Spring Boot tech stack, the integration cost is nearly zero.

Second, the stability of Spring AI after the 1.0 GA release is noticeably better. I tried the 0.8 version before; the API changed too frequently, and every upgrade required changing a bunch of code. After 1.0, the API is stable, the documentation is complete, and I feel confident using it in production.

Of course, if your project isn't on the Spring Boot tech stack, LangChain4j might be more suitable. There's no absolute right or wrong in technology choices; just pick what fits your team.

Next Steps

Currently, this AI agent is still a relatively simple "single-round retrieval + function calling" model. A few directions I want to explore next:

State management for multi-turn conversations—for example, when a user says "I want to return an item", the AI can guide the user through the entire return process, rather than just a single Q&A
Integrating more Functions—like directly helping users create return orders or query detailed logistics information
Automated performance evaluation—currently, accuracy is checked by manual sampling; I want to build an automated evaluation system

If anyone else is working on something similar, feel free to reach out. Honestly, this field is changing too fast; Spring AI releases a new version almost every two weeks, and I'm continuously learning too.

If this article was somewhat useful to you, please give it a follow. Not for any other reason, just to document the process of stumbling through pitfalls, so that if someone else stumbles into the same ones, they can at least avoid some detours.

"Curly's Tech Notes", the real technical notes of a 9-year Java developer.