Backend · Frontend · AI Programming

34 Core Concepts That Take You From LLM Basics to Production Agent Systems

By 前端小小栈 · Aug 7, 2026

Read original on juejin.cn ↗ Google Translate ↗ Alt translation

Agent development has moved past demo stage into production engineering, and the vocabulary has exploded with it. A developer who confuses ReAct with Reflexion, or skips hybrid search in their RAG pipeline, will ship a system that breaks in ways that are expensive to debug and hard to explain to users.

Summary

Building a production AI agent demands fluency across a sprawling stack of concepts that didn't exist three years ago. This reference walks through 34 of them, organized from the ground up: how LLMs consume tokens and context windows, why chain-of-thought prompting improves accuracy at a cost, and what separates a single-shot LLM call from an autonomous agent running a ReAct loop.

Memory and retrieval get their own deep treatment, distinguishing short-term from long-term memory along both temporal and content axes, and laying out the full RAG pipeline from chunking and embedding to hybrid search and reranking. The guide also maps the emerging protocol layer—MCP for tool connection, A2A for agent-to-agent communication—and the Claude Code ecosystem of skills, hooks, and subagents.

Production concerns round out the picture: evaluation frameworks, observability, prompt caching economics, and the security risks of prompt injection and the lethal trifecta. The final section frames the whole discipline as a three-layer engineering problem—prompt engineering, context engineering, and harness engineering—where the model itself is just one component in a larger control system.

Takeaways

— An LLM is a pure function—input prompt to output text—with no built-in memory, internet access, or agency; everything beyond that is external engineering.

— Tokens are the billing and capacity unit; Chinese text costs roughly 1.5–2 tokens per character, and prompt caching can cut costs by 50–90% on repeated prefixes.

— Chain-of-Thought prompting improves accuracy but increases output tokens, making responses slower and more expensive.

— An Agent is defined by three elements running in a loop: an LLM for reasoning, tools for action, and a perception-decision-action-observation cycle.

— ReAct is one specific agent pattern (Thought → Action → Observation), not a synonym for Agent; CodeAct and planning agents are separate patterns.

— RAG is a two-stage pipeline: ingest documents via chunking and embedding into a vector database, then query with semantic search, hybrid search, and reranking before feeding results to the LLM.

— Memory splits along two axes: short-term vs. long-term (temporal) and working, episodic, semantic, or procedural (content type per the CoALA framework).

— Reflexion differs from Self-Refine by requiring a persistent episodic memory store that accumulates lessons across trials.

— MCP standardizes three primitives—Tools, Resources, Prompts—as a server-client protocol, functioning as a USB interface for LLMs.

— Production agents need evaluation frameworks, observability logging, and guardrails; without them, an agent is untested code running with access to tools and data.

— Prompt injection exploits the LLM's inability to distinguish system instructions from data; the lethal trifecta (private data access + untrusted content + external communication) makes exfiltration possible.

— Agent engineering is a three-layer stack: prompt engineering (strings), context engineering (information packing), and harness engineering (execution control, retries, permissions, circuit breakers).

Conclusions

The guide's framing of LLMs as pure functions is a useful corrective to anthropomorphic thinking that leads engineers to over-trust model outputs or assume persistence that doesn't exist.

Positioning Self-Refine as a sibling pattern to ReAct rather than a separate category clarifies why many coding tools can implement reflection-like behavior without a full Reflexion memory store.

The distinction between Contextual Retrieval (Anthropic's chunk-level summarization) and standard RAG is subtle but important—it addresses the 'lost meaning' problem that chunking creates, and is still under-adopted in practice.

Simon Willison's lethal trifecta is a concrete security model that gives teams a clear checklist: if your agent has all three capabilities, you have a data exfiltration vector that prompt injection can exploit.

The three-layer engineering model (prompt, context, harness) reframes agent development as a systems engineering discipline rather than a prompting art, which matches where production teams actually spend their time.

Concepts & terms

Token

The sub-word unit LLMs process; billing and context windows are measured in tokens. Chinese characters cost roughly 1.5–2 tokens each, English words about 1.3 tokens.

Context Window

The maximum number of tokens an LLM can process in one request. Models in 2026 reach 1–2 million tokens, but performance degrades in the middle of long contexts ('Lost in the Middle').

Chain-of-Thought (CoT)

A prompting technique that makes the LLM output reasoning steps before the final answer, improving accuracy at the cost of higher token usage and latency.

Agent

A system combining an LLM, tools, and a continuous loop (perceive → decide → act → observe) that runs autonomously until a goal is met or resources are exhausted.

ReAct

A specific agent pattern cycling through Thought, Action, and Observation steps. It is one agent pattern among several, not a synonym for Agent.

RAG (Retrieval-Augmented Generation)

A two-stage architecture that ingests documents into a vector database, then retrieves relevant chunks at query time to augment the LLM's prompt with external knowledge.

Embedding

A numerical vector representation of text or images where semantically similar items are positioned closer together in vector space, enabling similarity search.

Hybrid Search

Combining semantic (embedding-based) search with keyword (BM25) search, then merging and ranking results—the default approach for production RAG systems.

MCP (Model Context Protocol)

An open protocol standardizing how LLMs connect to external tools, data, and prompt templates via a server-client architecture, analogous to a USB interface for AI models.

Prompt Injection

An attack where malicious instructions are hidden in data the LLM processes, exploiting its inability to distinguish system commands from user-supplied content.

Lethal Trifecta

Simon Willison's security model: when an agent has access to private data, exposure to untrusted content, and external communication capability, prompt injection can enable data exfiltration.

Harness Engineering

The execution and control layer surrounding an LLM—agent loops, tool registration, retries, permissions, circuit breakers—everything that is neither model weights nor prompts.

Source: juejin.cn ↗ Google Translate ↗ Backup ↗