Frontend · Artificial Intelligence · AI Programming

The Architecture Terms Behind China's Latest Open-Source LLMs, Explained Plainly

By 恋猫de小郭 · Aug 30, 2026

Read original on juejin.cn ↗ Google Translate ↗ Alt translation

These architecture choices directly determine whether a 1M-context model fits on consumer hardware, runs at interactive speed, and retains early-context accuracy. Understanding the terms lets an engineer read a model card and immediately judge VRAM requirements, inference throughput, and the likely quality of long-context retrieval.

Summary

The recent wave of Chinese open-source models—GLM 5.3 Flash, Qwen 3.8 Flash, Hy4 Preview, Kimi K3—ships with a dense vocabulary: MoE, Top-8, GQA, MLA, DSA, IndexCache, iHC, mHC, GR, AttnRes, KDA, GDN, QSA, CSA, HCA, and a zoo of quantization labels. Each term addresses a specific cost bottleneck that appears when context windows hit 1M tokens and parameter counts reach hundreds of billions.

Attention variants (GQA, MLA, DSA, CSA, HCA) all attack the same problem from different angles: how to retrieve relevant history without storing or scanning every token. Linear-attention hybrids (KDA, GDN) compress history into a fixed-size recurrent state, then periodically fall back to sparse or standard attention for precision. Residual-stream upgrades (iHC, mHC, GR, AttnRes) prevent early-layer information from being diluted across 80+ layers by maintaining multiple parallel state highways or letting deeper layers dynamically pull representations from earlier stages.

On the quantization side, the article disentangles BF16/FP8/INT4 (how a single number is stored), MXFP8/NVFP4/W4A16 (how groups of low-precision numbers share a scale), and GPTQ/AWQ/GGUF recipe names (how the model is compressed without destroying capability). A 770B-parameter Hy4 in BF16 would occupy roughly 1.54 TB; aggressive mixed quantization brings a working GGUF down to ~200 GiB with negligible quality loss.

Takeaways

— MoE with Top-8 routing keeps total parameter count high (770B) while activating only ~49B parameters per token, controlling per-token compute.

— GQA cuts KV Cache size by having multiple query heads share one set of Key/Value tensors; MLA compresses K/V further into a 512-dimension latent vector.

— DSA uses a lightweight Indexer to score 1M tokens and selects the top 2048 for full attention, dropping complexity from O(L²) to O(L·k).

— IndexCache reuses the Indexer’s top-k list across adjacent layers, eliminating ~73% of Indexer runs in Hy4’s 78-layer stack.

— iHC maintains four parallel 6144-dim residual streams per token; each sub-layer dynamically reads from and writes to them with learned weights.

— AttnRes lets a deep layer attend to representations from earlier layers, pulling critical early signals that fixed residual accumulation would dilute.

— Linear-attention hybrids (KDA/ GD N) maintain a fixed-size recurrent state and periodically invoke sparse or full attention for precision—Qwen 3.8 Flash cycles 3 GDN + 1 QSA, Kimi K3 cycles 3 KDA + 1 Gated MLA.

— CSA compresses every ~4 historical tokens into one KV entry before the Indexer runs; HCA compresses ~128 tokens into one entry and drops the Indexer entirely, attending over all compressed blocks.

— Quantization splits into three layers: number format (BF16/FP8/INT4), block scaling (MXFP8/NVFP4/W4A16), and compression algorithm (GPTQ/AWQ/SmoothQuant/GGUF recipes).

— A 770B BF16 model theoretically needs ~1.54 TB; Hy4’s MIX-STQ1_0 GGUF fits in ~200 GiB by applying STQ1_0/IQ2 to routed experts while keeping attention and router layers at Q5–Q8 or FP32.

Conclusions

The simultaneous appearance of iHC, GR, and AttnRes across Hy4, Qwen, and Kimi K3 suggests multi-stream residuals are becoming a standard answer to depth-related information decay, not a single-lab experiment.

IndexCache and IndexPool reveal that even the ‘cheap’ Indexer in sparse attention is now considered too expensive, pushing compression into the search step itself.

The convergence on Linear + Sparse hybrid architectures (3+1 layer cycling) across three independent teams implies pure attention over full contexts is no longer viable at 1M-token scale.

GGUF quantization recipe names (Q4_K_M, IQ3_XXS, UD-Q4_K_XL) encode community-evolved mixed-precision strategies, not uniform bit-widths—treating them as simple 4-bit labels underestimates actual file size and quality.

Concepts & terms

MoE (Mixture of Experts)

Replaces a single FFN with many parallel ‘expert’ sub-networks; a Router scores each token and sends it to only the top-k experts, keeping total parameters large while limiting per-token compute.

GQA (Grouped Query Attention)

Multiple query heads share one set of Key/Value tensors, reducing the number of stored KV pairs and cutting memory bandwidth for long contexts.

MLA (Multi-head Latent Attention)

Compresses each token’s multi-head Key/Value information into a low-dimensional latent vector (e.g., 512-d), then reconstructs what is needed during attention, drastically shrinking the KV Cache.

DSA (DeepSeek Sparse Attention)

A lightweight Indexer scores all historical tokens and selects a small top-k subset (e.g., 2048) for full attention computation, reducing complexity from quadratic to linear in context length.

IndexCache

Reuses the top-k token index computed by a nearby full Indexer layer across several subsequent layers, avoiding redundant index scans—Hy4 saves ~73% of Indexer runs this way.

iHC (identity Hyper-Connections)

Maintains multiple parallel residual streams per token; each sub-layer dynamically reads a weighted blend of streams and writes results back with separate per-stream weights, preserving distinct information pathways across depth.

AttnRes (Attention Residuals)

Instead of fixed residual accumulation, a deep layer performs attention over representations from earlier layers or blocks, letting the model pull critical early signals that later updates might otherwise dilute.

KDA (Kimi Delta Attention)

A linear attention variant that maintains a fixed-size recurrent state, updating it per token with fine-grained, per-channel decay rates so the model learns what to retain, overwrite, or forget.

CSA / HCA (Compressed / Heavily Compressed Attention)

CSA compresses ~4 tokens into one KV entry before sparse indexing; HCA compresses ~128 tokens into one entry and attends over all compressed blocks directly, dropping the Indexer entirely.

MXFP8 / NVFP4

Block-scaled low-precision formats: MXFP8 groups 32 elements under one power-of-2 scale; NVFP4 groups 16 elements under a finer FP8-scale, trading slightly more bits for better fidelity.

GGUF quantization recipe (Q4_K_M, IQ3_XXS, etc.)

Community preset names encoding mixed-precision strategies—most weights use low bits while sensitive tensors (attention, routers) stay at higher precision. The number indicates an approximate bit-width tier, not a uniform format.

Source: juejin.cn ↗ Google Translate ↗ Backup ↗