跪拜 Guibai
← All articles
DeepSeek · AI Coding · AIGC

DeepSeek V4 Pro Lands with 1.6T Parameters, 1M Context, and a Coding Edge

By ServBay ·
Read original on juejin.cn ↗ Google Translate ↗ Alt translation

A 1M-token context window that actually runs at 27% of the compute cost of the previous generation changes the economics of long-document processing and multi-step agent workflows. Coding benchmarks that beat GPT-5.4 and Gemini 3.1 Pro on LiveCodeBench and Codeforces, at a fraction of the output price, make V4 Pro a credible default for code-generation pipelines.

Summary

DeepSeek V4 Pro uses a Mixture-of-Experts architecture with 1.6T total parameters and 49B activated, supporting a 1M-token context window and up to 384K tokens of output. A hybrid attention design combining compressed sparse and heavily compressed attention cuts single-token inference FLOPs to 27% of V3.2 and KV cache usage to 10%, making million-token contexts practical rather than theoretical. The model also adopts manifold-constrained hyper-connections for training stability at scale and the Muon optimizer for faster convergence across 32T+ pre-training tokens.

On LiveCodeBench, V4 Pro Max scores 93.5 versus Gemini 3.1 Pro's 91.7, and its Codeforces rating of 3206 edges out GPT-5.4 xHigh at 3168. Agent benchmarks put it in the same tier as Opus 4.6 and Gemini 3.1 Pro, with an 80.6% solve rate on SWE Verified. Three inference modes—Non-Think, Think High, and Think Max—let developers trade speed for depth depending on the task.

Pricing starts at 0.025 yuan per million cached input tokens and 6 yuan per million output tokens, with a concurrency cap of 500. The API supports JSON structured output, tool calls, and both OpenAI and Anthropic protocol formats, so existing integrations can switch by changing a base URL and key.

Takeaways
— V4 Pro is a 1.6T-parameter MoE model with 49B activated parameters, a 1M-token context window, and a 384K-token max output length.
— Hybrid attention (CSA + HCA) reduces single-token inference FLOPs to 27% of V3.2 and KV cache usage to 10% at 1M-token context.
— Manifold-constrained hyper-connections replace standard residual connections to stabilize gradient propagation in the 1.6T-parameter model.
— The Muon optimizer improves convergence speed and training stability over 32T+ pre-training tokens.
— Post-training uses a two-stage process: independent domain-expert cultivation via SFT and RL, then unification through on-policy distillation.
— Three inference modes are available: Non-Think for fast responses, Think High for logical reasoning, and Think Max for competition-level depth.
— V4 Pro Max scores 93.5 on LiveCodeBench and a 3206 Codeforces rating, ahead of GPT-5.4 xHigh and Gemini 3.1 Pro on coding.
— Agent benchmarks show an 80.6% solve rate on SWE Verified and 73.6 on MCPAtlas Public, matching Opus 4.6 and Gemini 3.1 Pro.
— Pricing is 0.025 yuan per million cached input tokens, 3 yuan per million uncached input tokens, and 6 yuan per million output tokens.
— Concurrency is capped at 500 for V4 Pro versus 2,500 for V4 Flash, making Flash the choice for high-frequency, lower-stakes calls.
— The API supports JSON output, tool calls, and both OpenAI and Anthropic protocol formats for drop-in replacement.
Conclusions

Cutting KV cache to 10% of the previous generation at full 1M context is the kind of efficiency gain that turns a spec-sheet number into a usable feature; most long-context claims still carry prohibitive latency and memory costs.

V4 Pro's coding lead over GPT-5.4 and Gemini 3.1 Pro on LiveCodeBench and Codeforces, combined with output pricing roughly an order of magnitude cheaper, pressures Western labs on the price-performance ratio for developer tools.

The two-stage post-training strategy—training domain experts separately, then distilling them into one model—sidesteps the interference that makes joint multi-task fine-tuning brittle, and it explains how a single model can stay competitive across coding, math, and agent tasks simultaneously.

Offering three explicit reasoning modes with documented response formats gives developers a predictable cost-latency-quality tradeoff, which is more operationally useful than a single opaque 'smart' mode.

Concepts & terms
Mixture-of-Experts (MoE)
A model architecture where only a subset of parameters (experts) are activated for a given input, allowing much larger total parameter counts without proportionally increasing compute per token.
Compressed Sparse Attention / Heavily Compressed Attention (CSA + HCA)
A hybrid attention mechanism that reduces the computational and memory cost of processing long contexts by compressing parts of the attention matrix, lowering FLOPs and KV cache usage.
Manifold-Constrained Hyper-Connections (mHC)
A replacement for standard residual connections that constrains signal propagation to a manifold, improving training stability in very deep or large models.
Muon Optimizer
An optimizer that improves convergence speed and training stability compared to traditional optimizers like Adam, used here for pre-training a 1.6T-parameter model on 32T+ tokens.
On-Policy Distillation
A technique where knowledge from separately trained expert models is transferred into a single unified model using data generated by the experts' own policies, preserving domain-specific capabilities.
Source: juejin.cn ↗ Google Translate ↗ Backup ↗