Artificial Intelligence

The LLM Lexicon, Part 3: 30 More Core Concepts, Illustrated

By 周末程序猿 · Jul 24, 2026

Read original on juejin.cn ↗ Google Translate ↗ Alt translation

LLM terminology has become a prerequisite for reading any AI engineering paper or system design doc. A visual glossary that connects terms like "speculative decoding" or "flash attention" to their mechanical purpose cuts through the jargon and gives engineers a faster on-ramp to optimization and deployment decisions.

Summary

The third installment of an illustrated glossary maps out 30 additional core concepts in large language models. Each entry pairs a concise definition with a diagram, covering the machinery that makes modern LLMs tick: speculative decoding, flash attention, mixture of experts, and quantization techniques like GPTQ and AWQ.

Agent-related concepts get significant attention, including ReAct, Plan-and-Solve, and multi-agent collaboration patterns. The glossary also addresses safety and alignment through RLHF, constitutional AI, and red teaming, alongside practical deployment concerns like KV cache management and continuous batching.

The series functions as a visual reference for the sprawling LLM terminology landscape. This batch spans the operational middle ground between raw architecture and end-user applications, clarifying how models are made faster, safer, and more capable of tool use.

Takeaways

— Speculative decoding uses a small draft model to propose tokens that a larger model verifies in parallel, cutting latency without changing output distribution.

— Flash attention restructures the attention computation to minimize HBM reads and writes, delivering 2-4x speedups and enabling longer context windows.

— Mixture of Experts (MoE) activates only a subset of parameters per token, allowing models to scale total parameter count without proportionally increasing compute per inference.

— GPTQ and AWQ are post-training quantization methods that compress model weights to 4 bits with minimal accuracy loss, making large models practical on consumer GPUs.

— KV cache stores previously computed key-value pairs to avoid recomputation during autoregressive generation, but its memory footprint grows linearly with sequence length.

— Continuous batching dynamically adds and removes requests from a running batch, maximizing GPU utilization for LLM serving.

— ReAct interleaves reasoning traces with tool-use actions, giving an agent a structured loop for deciding when to think and when to act.

— RLHF trains a reward model on human preference data and uses it to fine-tune the LLM via reinforcement learning, aligning outputs with human values.

— Constitutional AI replaces human feedback with a set of written principles that the model uses to critique and revise its own outputs.

— Red teaming systematically probes a model for harmful outputs, jailbreaks, and biases before deployment.

Conclusions

The glossary's structure reveals a field that has split into three distinct engineering disciplines: inference optimization (making models fast), agent orchestration (making models act), and safety alignment (making models behave). Each now has its own deep sub-vocabulary.

Quantization methods like GPTQ and AWQ have quietly become the unsung enablers of the local LLM boom. Without them, running a 70B-parameter model on a single GPU would remain a datacenter-only proposition.

Agent architectures like ReAct and Plan-and-Solve are converging on a common pattern: an explicit separation of reasoning from execution, with the model acting as its own supervisor. This mirrors how human experts work and suggests the next wave of agent frameworks will bake this split into their core abstractions.

Concepts & terms

Speculative Decoding

A two-model inference technique where a small, fast draft model generates candidate tokens and a large target model verifies them in parallel, accepting or rejecting each. It preserves the target model's output distribution while reducing latency.

Flash Attention

An IO-aware attention algorithm that tiles the attention matrix computation to minimize reads and writes to GPU high-bandwidth memory. It reduces the quadratic memory cost of standard attention to linear in practice.

Mixture of Experts (MoE)

A model architecture where only a fraction of the total parameters (the "experts") are activated for any given input token, selected by a gating network. This decouples total parameter count from per-token compute cost.

KV Cache

A memory buffer that stores the key and value tensors from previous tokens during autoregressive generation, so the model does not recompute them for each new token. Its size grows linearly with sequence length and batch size.

RLHF (Reinforcement Learning from Human Feedback)

A three-stage alignment process: collect human preference comparisons between model outputs, train a reward model on those preferences, then fine-tune the LLM with reinforcement learning to maximize the reward model's score.

Constitutional AI

An alignment method where a model self-improves by generating responses, critiquing them against a written constitution of principles, and revising them. The revised outputs are then used for supervised fine-tuning and RL from AI feedback.

Source: juejin.cn ↗ Google Translate ↗ Backup ↗