← Back to the summary

The 12 AI Terms That Keep Tripping Up Backend Devs (and the Architecture That Ties Them Together)

Suitable for: backend developers who know Spring Boot and want to get started with AI application development, but are put off by a pile of new terms. What you'll get: a memorizable architecture diagram, a concept comparison table, a minimal runnable code skeleton, and 5 pitfalls I actually stepped in.

1. Self-test first: how many of these 5 questions can you answer?

Don't look anything up, answer from intuition:

The "2 routes, 3 methods, 4 call types" for LLM integration — what do they each mean?
What is the relationship between "Bailian" and "Lingji DashScope"? Which one should you use when writing code?
In RAG, which of the ETL and retrieval steps run offline, and which runs on every request?
What's the difference between Tool Calling and MCP? Why do we need the latter if we have the former?
When the model calls a tool, why do the logs show two rounds of Request/Response?

If you can't answer 2 of these 5, this article is for you.

The first time I looked at these terms, my biggest feeling was: each word makes sense on its own, but put together they get tangled. Later I found the root of the tangle isn't technical — it's that the conceptual layering isn't clear. The same thing has a different set of names depending on whether you look at it from "where it's deployed," "how you touch it," or "how you write code." Mix them up and you get knots.

So this article doesn't follow tutorial chapters. It's rearranged by cognitive order: first see the whole architecture, then break it into four capability blocks, and finally land in code.

2. One diagram to understand the layered architecture of an AI application

Any AI application's skeleton is these five layers:

Four key points, memorize them and you can draw it yourself:

1) Advisor is AOP. Pre-Advisors act before the request enters the model (sensitive-word filtering, prompt assembly, injecting retrieval results). Post-Advisors act after the response leaves the model (persisting memory, logging, desensitization). Multiple Advisors are ordered with @Order.

This is the most valuable abstraction of Spring AI compared to other frameworks — all the cross-cutting logic you'd want to add can hang on an Advisor, and your business code doesn't change.

2) The four capability blocks in the middle are not peer-level calls; their trigger methods are completely different.

Capability	Who decides whether to use it	How to configure in code
Tools	The model decides itself (passive trigger)	`.tools(new MyTools())`
MCP	The model decides itself (remote process)	Configure MCP Client
RAG	The framework actively stuffs it into the Prompt	Attach `QuestionAnswerAdvisor`
Memory	The framework automatically injects history	Attach `MessageChatMemoryAdvisor`

Confuse these four and you'll write code like "I configured the knowledge base but the model doesn't query it" — because you thought it was automatic, but it's actually passive.

3) One question's real chain is 9 steps, and step 7 is a loop.

sequenceDiagram
    participant U as User
    participant S as Server
    participant M as LLM
    participant T as Tool/Vector store

    U->>S: ask
    S->>S: pre-Advisors -> get memory -> RAG retrieval
    S->>M: request with tool definitions
    M-->>S: I want to call getWeather(Shanghai)
    S->>T: reflectively invoke @Tool method
    T-->>S: return result
    S->>M: feed result back, ask again
    M-->>S: final answer
    S->>U: SSE stream output

Key point: after the model returns tool_calls, Spring AI reflectively calls your method, then sends another request to the model with the result. So tool calling is not one request; it's a back-and-forth of "model → tool → model." Seeing multiple rounds of Request/Response in the logs is normal, not a bug.

4) External dependency checklist = the environment you need to prepare. Vector store (PGvector), model API or local Ollama, file system (memory persistence / PDF generation), and — the terminal. Giving the model terminal access is equivalent to giving it the ability to execute commands; this is a security red line, and we'll return to it in the MCP section.

3. RAG: two diagrams say the same thing

RAG is the thing you should memorize most of all, but online explanations are a mess: business articles draw a flow diagram, Spring AI docs draw an ETL diagram, and they look like two different things.

It's actually the same thing under two naming conventions:

Spring AI component	Stage	What it does	Business-perspective term
`Reader`	Offline	Reads data source, produces `Document`	Document collection
`Transformer`	Offline	Splits and cleans into chunks	Document chunking
`Writer`	Offline	Vectorizes and writes to store	Vector conversion and storage
`VectorStore`	Shared	Vector database (PGvector)	Vector database
`Retriever`	Online	Retrieves chunks with the question	Similarity search + TopK
`Augmenter`	Online	Combines question + chunks into new prompt	Query augmentation
`ChatModel`	Online	Generates the final answer	LLM

The online/offline line must be clear. ETL runs once when building the store; retrieval runs on every question. People who can't distinguish them write code like "re-chunk the document on every question" — I've seen more than one.

Another common misunderstanding: vector recall and conditional filtering are not two sequential things, but two parallel paths (hybrid search), and they merge and rank afterward. The two branch arrows in business diagrams refer to this.

Finally, RAG's "anti-hallucination triple set" — you can copy directly into your prompt:

Context information is below:
---------------------
{context}
---------------------
Given the context and provided history information and not prior knowledge,
reply to the user comment. If the answer is not in the context,
inform the user that you can't answer the question.

Three sentences, three effects: rely only on context, include the conversation history, say you can't answer if you can't. The third sentence is the most overlooked, but it's precisely the most effective at suppressing hallucinations.

4. The three most confusable distinctions (screenshot these)

Distinction 1: 2 routes / 3 methods / 4 call types

These three numbers aren't on the same dimension; mixing them leads to chaos:

Phrase	What question it answers	Specifically
2 routes	Whose machine does the model run on?	Cloud service / self-hosted
3 methods	What form do you use to touch it?	AI application platform / AI software client / write code
4 call types	How do you write it in code?	Platform SDK / HTTP direct / Spring AI / LangChain4j

String it together: You use "write code" as the method, through the "cloud service" route, calling the model with Spring AI.

How to choose the route? For individual developers, cloud service is enough; the compute and maintenance cost of self-hosting will scare you off. Enterprises that demand data not leave the network should consider self-hosting.

Distinction 2: Bailian vs. Lingji DashScope

Both names carry "Alibaba Cloud," and many people think they're the same thing:

	Bailian	Lingji DashScope
Positioning	Visual platform	Model API interface
Audience	Technical + non-technical, click the UI to build apps	Technical developers
Level	Higher-level, simpler	Lower-level, more flexible
One sentence	Click it out	Write it out

Conclusion: when writing code, you connect to Lingji. Spring AI's spring.ai.dashscope config points to Lingji; apps built in Bailian have a different calling method (via appId).

Distinction 3: Tool Calling vs. MCP

Tool Calling solves: how the model calls a function (protocol, parameters, result feed-back);
MCP solves: where those functions come from (unified standard, like USB-C, tools and data sources plug-and-play).

We need MCP even with Tool Calling, just like we need a "hand" and a "standard interface" — otherwise, every time you hook up a new tool, you'd rewrite the adapter for each framework.

5. The minimal runnable skeleton (Spring AI 1.0)

Theory done; here's the code. Note: 1.0 GA renamed all the starters — many people trip on the first step right here (pitfall 1 detailed below).

1. Dependencies

<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>org.springframework.ai</groupId>
      <artifactId>spring-ai-bom</artifactId>
      <version>1.0.0</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>

<dependencies>

  <dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-starter-model-openai</artifactId>
  </dependency>

  <dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-starter-vector-store-pgvector</artifactId>
  </dependency>
</dependencies>

2. Configuration

spring:
  ai:
    openai:
      api-key: ${AI_API_KEY}                       # Don't hardcode; use environment variable
      base-url: https://dashscope.aliyuncs.com/compatible-mode   # Tongyi compatible address
      chat:
        options:
          model: qwen-plus                          # Change model by editing only this line
          temperature: 0.7

Tongyi's compatible address ends with /compatible-mode, not /v1 — adding /v1 directly gives a 404. I stepped in this pit.

3. ChatClient: hook up memory and logging

@Configuration
public class AiConfig {

    @Bean
    public ChatMemory chatMemory() {
        // In 1.0 GA, InMemoryChatMemory is gone; use MessageWindowChatMemory
        return MessageWindowChatMemory.builder()
                .maxMessages(20)   // sliding window to prevent token explosion
                .build();
    }

    @Bean
    public ChatClient chatClient(ChatClient.Builder builder, ChatMemory memory, VectorStore vectorStore) {
        return builder
                .defaultSystem("You are a patient emotional counselor. Your answers should be concrete and actionable.")
                .defaultAdvisors(
                        MessageChatMemoryAdvisor.builder(memory).build(),  // automatic memory
                        new SimpleLoggerAdvisor()                          // log request/response
                )
                .build();
    }
}

Note MessageChatMemoryAdvisor.builder(...) is a builder form, not new(...) — the old tutorial's style won't compile on GA.

4. A conversation with memory

@RestController
@RequestMapping("/ai")
@RequiredArgsConstructor
public class ChatController {

    private final ChatClient chatClient;

    @GetMapping("/chat")
    public Flux<String> chat(@RequestParam String message,
                             @RequestParam String conversationId) {
        return chatClient.prompt()
                .user(message)
                .advisors(a -> a.param(ChatMemory.CONVERSATION_ID, conversationId))
                .stream()          // streaming, combined with frontend SSE for typewriter effect
                .content();
    }
}

5. Attach RAG

@Bean
public ChatClient ragChatClient(ChatClient.Builder builder, VectorStore vectorStore) {
    return builder
            .defaultAdvisors(
                    QuestionAnswerAdvisor.builder(vectorStore)
                            .searchRequest(SearchRequest.builder()
                                    .topK(5)                 // take the 5 most relevant
                                    .similarityThreshold(0.6) // below 0.6, drop it
                                    .build())
                            .build())
            .build();
}

Ingestion (offline, run once):

@Component
@RequiredArgsConstructor
public class DocIngestService {

    private final VectorStore vectorStore;

    public void ingest(Resource resource) {
        List<Document> raw = new TextReader(resource).get();
        List<Document> chunks = new TokenTextSplitter().apply(raw);  // chunk
        vectorStore.add(chunks);                                     // vectorize + store
    }
}

6. Add a tool (Tool Calling)

public class WeatherTools {

    @Tool(description = "Query the real-time weather of the given city. The parameter is the Chinese city name, e.g., Shanghai")
    public String getWeather(@ToolParam(description = "City name") String city) {
        return city + ": sunny, 26°C, southeast wind level 3";
    }
}

String answer = chatClient.prompt()
        .user("Is it good to go out in Shanghai today?")
        .tools(new WeatherTools())   // attach; the model decides whether to call it
        .call()
        .content();

The key is the @Tool description. It's not a comment; it's the "user manual" for the model — if it's vague, the model won't call the tool, or will call it wrongly. This matters more than most people think: tool-calling accuracy is 80% dependent on how well the description is written.

7. Structured output

public record LoveReport(String score, List<String> problems, List<String> suggestions) {}

LoveReport report = chatClient.prompt()
        .user("Analyze this conversation for me: " + dialog)
        .call()
        .entity(LoveReport.class);   // auto-convert to JSON and deserialize

6. 5 pitfalls I actually stepped in

Pitfall 1: Spring AI 1.0 renamed all starters

90% of online tutorials still use the old coordinates; copying them directly causes "dependency not found":

Old coordinates (before 1.0)	New coordinates (1.0 GA)
`spring-ai-openai-spring-boot-starter`	`spring-ai-starter-model-openai`
`spring-ai-pgvector-store-spring-boot-starter`	`spring-ai-starter-vector-store-pgvector`
`spring-ai-mcp-server-spring-boot-starter`	`spring-ai-starter-mcp-server`

Remember the pattern: spring-ai-starter-{type}-{specific}. Model is model, vector store is vector-store, MCP is mcp.

Similarly, InMemoryChatMemory was replaced by MessageWindowChatMemory, and the construction of multiple Advisors changed from new to builder. If it doesn't compile, suspect the version first, not yourself.

Pitfall 2: Thinking tool calling only requests the model once

The first time I saw two sets of Request/Response in the logs, I thought it was an infinite loop. Actually it's by design: the model asks for a tool, the tool runs, then it asks again with the result.

What you need to guard against is the real infinite loop — in an agent you must set a maxSteps circuit breaker:

if (currentStep >= maxSteps) {
    state = AgentState.FINISHED;
    results.add("Terminated: Reached max steps (" + maxSteps + ")");
    return String.join("\n", results);
}

When you write your own agent, add this first, or one runaway run can burn half a month's quota.

Pitfall 3: Putting ETL in the request path

RAG's chunking and vectorization are offline. I've seen people re-read the document, re-chunk, and re-ingest in every chat endpoint — single response time goes from 1 second to 8 seconds, and you can't find the cause.

How to tell: Is this code executed on every request? If yes, and it involves "read file / split / embedding," then it's in the wrong place.

Pitfall 4: The ❌ in Spring AI's official capability table doesn't mean "the model can't"

The official docs have a Chat Models capability comparison table, and the DeepSeek row marks Tools with ❌. But DeepSeek's official API clearly supports function calling.

The reason is: the table refers to whether the Spring AI integration implements it, not whether the model itself can. When DeepSeek is accessed through an OpenAI-compatible proxy, the integration layer hasn't hooked up tool calling.

So when tools don't trigger, first switch to a natively integrated model (like Tongyi) to verify, don't rush to change your @Tool.

Pitfall 5: Adding `/v1` to Tongyi's compatible address

base-url: https://dashscope.aliyuncs.com/compatible-mode/v1 → direct 404.

The correct form ends at /compatible-mode. This kind of pit has no technical content, but it can stall you for half an hour.

7. How to choose a model: don't look at parameters, look at four things

Leaderboards are flashy; in practice, choose by four dimensions:

Dimension	The question to ask yourself
Accuracy	How high is your scenario's tolerance for errors? Can you accept 5% nonsense?
Feature support	Do you need image input (multimodal), tool calling (function calling), long documents (context window)?
Performance and cost	Unit price × daily call volume = monthly bill. Decide between Max and Plus after calculating.
Data compliance	Can data leave the network? If not, you must self-host an open-source model.

Official comparison entry points for selection:

Spring AI: docs.spring.io/spring-ai/reference/1.0/api/chat/comparison.html
LangChain4j: docs.langchain4j.dev/integrations/language-models/

One experience: for demos use the strongest model (saves time); for products, calculate cost first before choosing (otherwise you'll lose money once it's live).

8. Appendix: quick lookup table for the 12 terms (screenshot this)

The 12 terms from the title, compressed into one table. Come back anytime:

#	Term	One-sentence explanation	Easy-to-miss point
1	LLM	Deep network with billions to trillions of parameters; understands and generates language, processes multimodal data	Parameters aren't always better; cost must be calculated
2	Token	The smallest unit the model processes text in; roughly 1 Chinese character ≈ 1-2 tokens	Both billing and context length count tokens
3	Prompt	The input instruction to the model, with system / user / assistant roles	system sets the persona, user states the need
4	temperature / top_p	The former controls randomness (high = divergent), the latter is nucleus sampling (low = conservative)	Tuning both together causes conflicts; tune one first
5	Embedding	Converts text into a vector (a list of floats); semantically similar things are closer in distance	Changing the embedding model requires rebuilding the store
6	Vector database	A store dedicated to vectors and similarity search, e.g., PGvector	Index creation isn't always required upfront
7	Chunk	Splits long documents into small pieces; the smallest unit of retrieval	Too big → recall is inaccurate; too small → loses context
8	RAG	Retrieves material first, then asks the model to answer based on it; "open-book exam"	Offline building the store ≠ online retrieval
9	Rerank	Re-ranks the TopK results from initial recall; improves accuracy	It's fine-ranking, not a second recall
10	Tool Calling	The model autonomously decides to call your function and brings the result back to the conversation	Bad description → inaccurate calls
11	MCP	Standardized protocol for tools and data sources; the USB-C of AI	Not a replacement for Tool Calling
12	Agent / ReAct	An agent that breaks down tasks and loops "reason-act" until completion	Must set maxSteps circuit breaker

Plus three engineering concepts you'll bump into most when writing code:

Term	One sentence
Advisor	Spring AI's aspect; insert logic before/after requests (security, memory, RAG, logging)
ChatMemory	Storage of conversation history, isolated by conversationId, persistable
SSE	Server-side unidirectional streaming push; the frontend's "typewriter effect" depends on it

9. Final words

Looking back, what's really hard in AI app development isn't the model — it's hanging new concepts onto your existing knowledge system:

Advisor ≈ AOP aspects
RAG ≈ open-book exam (read the book first, then answer)
Tool Calling ≈ giving the model hands
MCP ≈ standard interface for the hands (USB-C)
Agent ≈ an intern who writes their own to-do list

Once you map it this way, everything left is engineering problems — and engineering is what backend developers are already good at.

Three conclusions that are easiest to remember:

Advisor is the entry point: all cross-cutting logic (security, memory, RAG, logging) hangs on it; business code stays uninvasive.
Offline stays offline: ETL builds the store once, retrieval runs every time; don't mix them.
Description is the interface: the @Tool description is like a prompt — it's "code meant for the model."

If this saved you a few hours, hit like and bookmark. I'll continue with RAG tuning (chunking parameters, TopK, Rerank real-comparison) and self-built Agent pitfall notes — that's where the real time goes.

Questions are welcome in the comments; I reply to all I see.