跪拜 Guibai
← All articles
Backend · Programmer · Artificial Intelligence

GPT-5.6 Lands With Three Models, but Early Benchmarks and Hands-On Testing Tell a Messier Story

By cxuanAI ·
Read original on juejin.cn ↗ Google Translate ↗ Alt translation

The launch resets the competitive landscape between OpenAI and Anthropic, but the benchmarks refuse a clean narrative. Developers choosing between GPT-5.6 and Fable 5 now face a genuine trade-off: coding-agent throughput and cost efficiency versus raw software-engineering accuracy and general reasoning. The gap between the top-tier Sol and the mid-tier Terra also means Plus subscribers may not experience the gains that Pro users see.

Summary

OpenAI launched GPT-5.6 in three tiers: Sol (Pro-only), Terra, and Luna, with all subscription quotas reset at launch. Sol adds Max and Ultra reasoning modes — Max extends deep thinking, Ultra spawns parallel sub-agents at high token cost. The old Codex is now ChatGPT Codex, joined by ChatGPT Work, a mode aimed at non-developers that hides technical details and can orchestrate tasks across Google Drive, Slack, and email.

Benchmarks paint a split picture. Sol scores 80 on a Coding Agent Index versus Fable 5's 77, and leads by 13.1 points on a 55-task agent workflow exam. But Fable 5 posts 80% on SWE-Bench to Sol's 64.6%, and edges Sol 60–59 on a composite intelligence index. OpenAI pushed back, claiming roughly 30% of SWE-Bench Pro questions are flawed.

Simon Willison's hands-on testing found GPT-5.6 no better than Fable 5 on complex coding tasks. A pelican-SVG generation grid across 18 model/reasoning combinations showed costs ranging from 0.71 cents (Luna, no reasoning) to 48.55 cents (Sol, Max). The author's own Terra Ultra tests produced an empty final message and mixed-language output, leaving the real-world verdict unsettled.

Takeaways
GPT-5.6 ships as three models: Sol (Pro 5x/20x only), Terra, and Luna, with all subscription quotas reset at launch.
Sol introduces Max reasoning (longer deep thinking) and Ultra reasoning (parallel sub-agents), with Ultra consuming significantly more tokens.
ChatGPT Codex replaces the old Codex, and ChatGPT Work debuts as a non-developer mode that hides technical details and orchestrates tasks across Google Drive, Slack, and email.
On a Coding Agent Index, Sol scores 80 versus Fable 5's 77; on a composite intelligence index, Fable 5 leads Sol 60–59.
Fable 5 scores 80% on SWE-Bench to Sol's 64.6%, though OpenAI claims about 30% of SWE-Bench Pro questions are flawed.
Simon Willison's 18-configuration pelican-SVG test shows costs ranging from 0.71 cents (Luna, no reasoning) to 48.55 cents (Sol, Max).
The author's Terra Ultra tests produced an empty final message and mixed Chinese-English output, with slower generation than GPT-5.5.
Conclusions

GPT-5.6's tiered launch creates a two-class experience: Sol's gains are locked behind Pro plans, while Plus users get Terra and Luna, which benchmark closer to or below Fable 5.

OpenAI's preemptive critique of SWE-Bench Pro — that 30% of questions are flawed — is a strategic move to reframe a benchmark where its flagship model trails by 15 percentage points.

The pelican-SVG cost grid exposes a 68x price spread between the cheapest and most expensive configuration, making reasoning-intensity selection a real economic decision rather than a technical afterthought.

ChatGPT Work signals OpenAI's intent to compete in the office-productivity agent space, directly overlapping with tools like Qoder, Trae, and WorkBuddy that already split into developer and business-facing modes.

Concepts & terms
SWE-Bench
A benchmark that evaluates language models on real-world software engineering tasks drawn from GitHub issues, measuring their ability to produce correct patches.
Coding Agent Index
A composite score aggregating performance on DeepSWE, Terminal-Bench, and SWE-Atlas-QnA, designed to measure a model's effectiveness as an autonomous coding agent.
Computer Use
A ChatGPT capability that lets the model interact with applications through a browser — clicking, typing, navigating tabs, and downloading files — to complete multi-step tasks across authenticated websites.
Source: juejin.cn ↗ Google Translate ↗ Backup ↗