跪拜 Guibai
← Back to the summary

34 Core Concepts That Take You From LLM Basics to Production Agent Systems

🚀 Complete Beginner's Guide to AI Agents: From LLMs to Production, 34 Core Concepts Every Newcomer Must Know

Master AI Agent basics in one go. After reading this, jargon like Token, RAG, CoT, MCP, and ReAct will never confuse you again.

📋 Table of Contents

  1. LLM Basics: The "Brain" of AI
  2. Prompt Engineering: The Art of Talking to AI
  3. Agent Core: From Q&A to Autonomous Action
  4. Memory & Retrieval: Giving AI a "Memory"
  5. Multi-Agent & Protocols: Team Collaboration
  6. Claude Code Ecosystem Overview
  7. Production Deployment: From Usable to Excellent
  8. Security & Cost: Red Lines You Cannot Ignore
  9. Engineering Mindset: Upgrading to a Three-Layer Architecture
  10. Summary & Learning Path

一、LLM Basics: The "Brain" of AI 🧠

1. LLM (Large Language Model)

Models like GPT, Claude, and Gemini that take text in and return text out. Core insight: An LLM itself is a pure function:

input prompt → output text

It cannot browse the web on its own, nor can it remember the last conversation—these all require external systems.

2. Token: AI's "Currency Unit" 💰

LLMs don't see "characters"; they see tokens (sub-word units):

LLM billing and context windows are both calculated in tokens. "1 million token context" ≈ 750,000 Chinese characters.

💡 Money-saving tip: When writing prompts, use English abbreviations instead of long Chinese phrases whenever possible. Fewer tokens mean faster responses and a thinner bill.

3. Context Window

How many tokens an LLM can "see" at once. Cutting-edge levels in 2026:

But note: Bigger is not always better. Beyond a certain length, LLMs suffer from "Lost in the Middle".

💡 Practical advice: Put the most important instructions at the beginning and end of the prompt, with supplementary material in the middle.

二、Prompt Engineering: The Art of Talking to AI 🎨

4. Basic Prompt Structure

5. Zero-shot / One-shot / Few-shot

The difference between these three terms lies only in how many examples you provide:

Type Number of Examples Applicable Scenarios
Zero-shot 0 Ask directly, no examples given
One-shot 1 input→output example Simple format guidance
Few-shot 2-5 examples Tasks with strict format requirements; accuracy improves significantly 
# Few-shot Example
Translate the following Chinese into English:
Chinese: 你好 → English: Hello
Chinese: 谢谢 → English: Thank you
Chinese: 再见 → English:

6. Chain-of-Thought (CoT)

Make the LLM "think first, then answer"—output the reasoning process first, then give the conclusion.

Two forms:

  1. Few-shot CoT: Include examples with reasoning steps in the prompt for the LLM to imitate
  2. Zero-shot CoT: Add "Let's think step by step" at the end of the prompt to trigger reasoning

⚠️ Cost: CoT increases the number of output tokens, meaning it's slower and more expensive, but the accuracy gain is usually worth it.

三、Agent Core: From Q&A to Autonomous Action 🤖

7. Agent

A system centered on an LLM that can run autonomously in a loop. Three core elements:

Element Function
LLM Reasoning / Planning / Decision-making
Actions Means to do things (call tools, write code, query databases, etc.)
Loop The heartbeat cycle—Perceive → Decide → Act → Observe → Repeat

Key distinction:

📌 ReAct is one type of Agent pattern, not the definition of Agent. CodeAct, computer-use, and planning agents are all Agents.

8. Tool Use / Function Calling

Allows the LLM to call external functions you define. The LLM returns JSON instead of free text:

{
  "function": "search_weather",
  "args": {
    "city": "Beijing"
  }
}

Your program executes the function, feeds the result back to the LLM, and it continues based on the result.

🔌 Think of the LLM as the brain; Tool Use is its hands, feet, and senses.

Note: Anthropic calls it "Tool Use", OpenAI calls it "Function Calling". The API schemas differ slightly; align them properly when writing cross-vendor SDKs.

9. ReAct (Reasoning + Acting)

The most classic Agent pattern:

Thought → Action (call tool) → Observation (observe result) → Thought ...

Loops until a final answer can be given. Most Agent frameworks implement this internally.

10. Structured Output

Forces the LLM to output according to a fixed schema (like JSON) instead of free text. All major APIs support this via a response_format parameter. Agent frameworks rely almost entirely on this to communicate with LLMs.

11. Self-Refine (Basic Reflection)

The Agent self-evaluates the previous round's output and modifies the next round's approach:

Actor produces answer → Critic finds problems → Actor sees feedback and answers again

Does not require a persistent memory layer; essentially a sibling pattern to ReAct. Tools like Cursor and Cline run variants of this every day.

四、Memory & Retrieval: Giving AI a "Memory" 🧠💾

12. Memory—Two Classification Axes

"Memory" is often conflated; there are actually 2 orthogonal classifications:

Temporal Axis:

Content Axis (CoALA Framework):

Type Meaning Example
Working Temporarily stored information Current task steps
Episodic Past experiences What preference the user mentioned last time
Semantic Factual knowledge Technical parameters of the company's product
Procedural How to do something Standard process for calling a specific API

💡 Long-term memory can simultaneously contain episodic + semantic + procedural memories.

13. RAG (Retrieval-Augmented Generation)

Solves the problem of LLMs not knowing your private / changing / outdated data.

Two-stage architecture:

Stage 1: Ingest (Build the database)

document → chunk → embed (vectorize) → store in Vector DB

Stage 2: Query

question → embed → semantic search → top-K chunks → stuff into Prompt → LLM answers

14. Embedding

Converts text/images into N-dimensional vectors, so things with "similar meaning" are closer together. Defaults to dense embedding (dense vectors). There are also sparse embeddings (BM25 / SPLADE, etc., matching based on literal tokens).

15. Vector DB (Vector Database)

A storage layer for storing and efficiently querying embeddings. Core capability = ANN (Approximate Nearest Neighbor search), hundreds of times faster than brute-force full scan.

Representatives: Pinecone, Chroma, Qdrant, Weaviate, pgvector.

16. Chunking

Cutting long files into small segments suitable for embedding (usually 200-1000 tokens). The cutting method directly affects RAG quality—too fine loses context, too long blurs relevance.

17. Hybrid Search

Semantic search + keyword search (BM25) used together, then merged and ranked. This is the default standard for production-grade RAG, usually more accurate than a single method.

18. Reranking

The first round of retrieval fetches top-50, then a more expensive but more accurate model (cross-encoder) re-ranks them into top-5 for the LLM. Representatives: Cohere Rerank, bge-reranker.

19. Contextual Retrieval

A method proposed by Anthropic in 2024—prepend each chunk with a "context summary of the entire document" before embedding, avoiding the problem of "this chunk makes no sense out of context".

20. Fine-tuning

Retraining the model with your own data, "burning" knowledge into the weights.

Approach Essence Suitable For Not Suitable For
RAG Stuff data into context at inference time, no weight changes Latest facts, private docs, frequently changing knowledge Extremely complex format requirements
Fine-tune Retrain model, knowledge burned into weights Stably learning a specific format/style/domain terminology Stuffing "latest facts" (will expire and hard to update)

💡 Golden Rule: In Agent scenarios, exhaust Prompt + RAG first; only consider Fine-tuning when that truly isn't enough.

21. Reflexion (Full Reflection)

Different from Self-Refine: Reflexion requires a persistent episodic memory store. After an Agent completes a task, it writes a reflection summary into memory; the next task starts by retrieving this into the Prompt. Accumulating lessons across trials is the essence of Reflexion.

五、Multi-Agent & Protocols: Team Collaboration 👥

22. Multi-Agent

Multiple Agents collaborating to complete tasks. Common patterns:

Handoff: One Agent transfers a task to another Agent, involving context passing and failure handling.

23. A2A (Agent-to-Agent Protocol)

An inter-Agent communication standard initiated by Google and governed by the Linux Foundation. Reached v1.0 in 2026; it is the sister standard to MCP (agent↔tool), used for agent ↔ agent communication.

六、Claude Code Ecosystem Overview 🛠️

24. MCP (Model Context Protocol)

An open protocol launched by Anthropic in 2024, donated to the Linux Foundation in late 2025. Think of it as "the USB interface for LLMs".

Standardizes 3 primitives:

Primitive Function
Tools Functions the LLM can call
Resources Data the LLM can read
Prompts Reusable Prompt templates

Architecture: Server/Client model. Servers expose via stdio (local) or Streamable HTTP (remote).

25. Skills / SKILL.md

Claude Code's "behavior packs". A Skill = a folder containing SKILL.md + optional reference files.

Key mechanism: Before processing each message, Claude Code scans the frontmatter descriptions of all Skills—if one matches the current context, it's automatically loaded. Therefore, how well the description is written directly determines whether the skill gets triggered. In practice, starting with "Use when ..." is most effective.

26. Other Practical Concepts

七、Production Deployment: From Usable to Excellent 📈

27. Eval (Evaluation Framework)

Run a set of test cases against an Agent to quantify accuracy / latency / cost. An Agent without Eval is like code without tests.

Common tools: promptfoo, LangSmith, Langfuse.

28. Observability

Log every step inside the Agent (which LLM call, which tool, what result). Allows replay when debugging bugs.

29. Prompt Caching

The LLM caches the prompt prefix; subsequent calls with the same prefix only pay the cheap cache hit price (Anthropic 90% off, OpenAI 50% off). Can save significant money in long context + repeated query scenarios.

30. Computer Use (Screen-level Agent)

Operates real desktop applications via screenshot → visual understanding → calculate coordinates → simulate keyboard/mouse. Does not rely on APIs; sees the screen like a human.

Representatives: Anthropic Claude Computer Use, OpenAI Codex desktop.

31. Browser Use (Web-page-level Agent)

Operates web pages, primarily using DOM-aware navigation (directly querying CSS selectors) + visual fallback when necessary.

Representative open-source: browser-use (★ 105k+).

八、Security & Cost: Red Lines You Cannot Ignore ☠️

32. Prompt Injection

Hiding malicious instructions in content the LLM will read (web pages, documents, tool returns), inducing it to ignore the original task.

Root cause: LLMs cannot distinguish between "system instructions" and "instructions smuggled in data".

Defense: Least privilege, isolate untrusted content, human review for high-risk actions.

33. Lethal Trifecta

Proposed by Simon Willison: When an Agent simultaneously possesses these three capabilities, it can be manipulated by prompt injection to steal and exfiltrate data:

  1. Access to private data
  2. Exposure to untrusted content
  3. Ability to communicate externally

Defense: Break at least one link (commonly: cut off external communication or isolate untrusted input).

34. Guardrails

A rule layer preventing LLMs from doing bad things—blocking prompt injection, PII leakage, harmful output, etc.

九、Engineering Mindset: Upgrading to a Three-Layer Architecture 🏗️

When you move from "tweaking prompts" to "building products", you need to understand the three-layer engineering division of labor:

Layer 1: Prompt Engineering

Engineering strings—how to write prompts to get better model output.

Layer 2: Context Engineering

Engineering information—what information goes into the window for each LLM call (RAG results, memory, tool definitions, conversation history). Karpathy calls it "the fine art of packing exactly the information useful for the next step into the window."

Layer 3: Harness Engineering

Engineering the execution and control layer—Agent loop, tool registration, context management, permissions, security, retries, circuit breakers... all the code that is neither model weights nor the prompt itself.

Simon Willison: Coding Agent = LLM + Harness

Extended concepts:

十、Summary & Learning Path 🗺️

Core Concept Quick Reference

Concept One-Liner Understanding
LLM Pure text function, input → output
Token AI's billing unit; Chinese approx 1.5-2 tokens/char
Context Window How much text AI can see at once; watch out for "Lost in the Middle"
Few-shot Give examples, AI learns faster
CoT Make AI think first then answer; accuracy↑ token↑
Agent LLM + Tools + Loop; can work autonomously
Tool Use Protocol for AI to call external functions
ReAct Classic loop: Think→Do→See→Think again
RAG Give AI an external knowledge base
Embedding Turn meaning into vectors
Vector DB Database for storing vectors and searching similar meanings
MCP USB interface for LLMs, unified connection to external tools
Harness Execution and control layer outside the model

Suggested Hands-On Path

Step 1: First, use Claude Code / Cursor to write a few Skills, experience Tool Use
    ↓
Step 2: Use LangChain to build a ReAct Agent, run through the loop
    ↓
Step 3: Connect a RAG Pipeline to the Agent, experience Context Engineering
    ↓
Step 4: Add Eval and Observability; only then is it an Agent "ready for production"
    ↓
Step 5: Try Multi-Agent collaboration, understand A2A and MCP protocols

💬 Final words: The AI Agent field is developing extremely fast. Today, in 2026, we have already evolved from "tweaking prompts" to "designing complete Harness engineering". But the fundamentals remain unchanged: understanding the nature of LLMs (pure functions, with windows, prone to hallucination) is the starting point for designing any Agent system.

If this article was helpful, feel free to like ⭐️ bookmark and share in the comments what confusions you've encountered while learning about Agents!