Gemini 4 Carbon Tops AI Index at 59 Points, Then Vaporizes: No Release Date, 11% Hallucinations
The model at the top of the leaderboard is not something you can buy right now, even if you're willing to pay.
On October 29, the third-party evaluation organization Artificial Analysis put Google's Gemini 4 Carbon at the top of its Intelligence Index v5 with 59 points, ahead of Claude Opus 5.5 and GPT-6 Astra. On Hacker News, the news gathered 338 upvotes and 247 comments, but the highest-voted discussion wasn't about how strong it is, but when it will finally be opened up. Like the previous generation Argon when it launched, Carbon is currently only pushed to selected users, and no public timeline exists at all. Even more interesting is a counter-intuitive detail buried in the data: the new model's hallucination rate rose from 4% to 11%, nearly tripling, yet when placed in the full comparison, it is still the second lowest. This article breaks down the official evaluation data to see how much this top-ranked model is really worth, and what you and I should do right now.
What the Leaderboard Looks Like: Four Firsts Across Nine Items
Look at the total score first. Artificial Analysis's Intelligence Index v5 scores models on nine evaluation items, and Carbon took 59 points to become the new leader. Among the nine sub-items, it won four firsts: HLE-Diamond, Terminal-Bench-Science, AA-Omniscience, and AutomationBench-AA, covering hard reasoning, scientific tasks, and knowledge reliability. This index is maintained by a third party, scores nine dimensions independently, and aims to avoid single-benchmark gaming. The evaluators' qualitative assessment is that it leads in complex reasoning, knowledge, and science tasks, with major improvements in coding and knowledge work, but it does not lead in every dimension.
Put plainly: Carbon is a champion with an uneven profile. It pulls ahead of rivals on brain-bending reasoning and science problems, but in coding and everyday knowledge work—the two scenarios engineers care about most—it has closed the gap but not yet overtaken. This structure resembles a student who scores full marks on science but barely passes English: overall first, but with weak spots. When choosing, you can't look at just the total score bar; you have to go through sub-scores, price, and availability item by item, which is exactly what the next sections do.
The reference point is its own predecessor Argon. At the same pricing tier, Argon's Intelligence Index was only 50, and Carbon pushed it up by 9 points. Same price, same product line, and a nearly 20% score increase in one generation. That rate of progress is the signal truly worth watching behind the leaderboard. Leaderboards go stale; progress curves don't lie. On the spec side, Carbon carries a 1-million-token context window, supports text, image, video, and voice input modes, and still outputs pure text. The flagship configuration hasn't been cut down.
Coding Closes to Within 3 Points of Claude
Coding is historically a weak spot for Google models, but this time the numbers are quite competitive. On Artificial Analysis's Coding Agent Index, Carbon with Google's own Antigravity CLI scored 59, only 3 points behind Claude Code with Claude Opus 5.5's 62. Looking at the breakdown, Carbon got 64% on Terminal-Bench 4.0, ranking second; that test has the model complete operational tasks in a real terminal environment. On FrontierCode v1.1, it got 51%, also behind Opus 5.5's 56%, but both are now at touching distance.
What does a 3-point gap mean? It's the first time a Google model has entered close-quarters combat on the coding leaderboard. The evaluators specifically noted that Google has historically lagged in this area, so the improvement stands out. Another detail worth highlighting: Carbon pairs with Google's own Antigravity command-line tool, while the top combination is Claude Code with Opus 5.5. Coding results depend not only on model ability but also on the tuning of the companion tooling. In other words, if you previously crossed Gemini off your shortlist because of its coding ability, that decision needs to be revisited. In the coding assistant race, there is genuinely one more option for the first time, and that one carries a 1-million-token context window.
However, the Hacker News comments offer testimony from the other side. Multiple users reported that Argon in the web interface fabricates content in large chunks; one asked it to summarize project progress and it literally invented an entire passage. Other comments pointed out that web interface hallucinations may stem from the outer wrapper not requiring the model to verify information, not necessarily the model in the API. How wide the gap between benchmark scores and real-world feel is will only be verified once Carbon opens up.
Hallucination Rate Nearly Triples Yet Is Still Second-Lowest Across the Board
This is the most dramatic number in the entire dataset. On AA-Omniscience v1.1, which measures knowledge reliability, Carbon scored 54, 6 points above Opus 5.5's 48 and 8 points above Argon's 46. Accuracy rose 10 percentage points from Argon's level to 58%. But the cost is written on the same line: the hallucination rate jumped from Argon's 4% to 11%. This test specifically asks questions at the edge of the model's knowledge boundary, checking whether it honestly admits it doesn't know or makes up an answer. It's not measuring raw knowledge; it's measuring how honest the model is about its own limits.
Looking at just the increase, it's alarming—nearly threefold. But put the full field on the table and it's a completely different story: Opus 5.5 and GPT-6 Astra both have a 50% hallucination rate. That means these two rivals fabricate half of their answers in knowledge questions, while Carbon is at one in nine. On an absolute scale, 11% is still the second lowest, second only to its own overly conservative Argon. The increase is scary, but the base is still excellent.
This tradeoff deserves deeper thought. Argon's 4% hallucination rate comes at the cost of a lot of refusals; the model doesn't dare answer, so it rarely errs. Carbon clearly chose a bolder answering path, with a big jump in accuracy and a corresponding rise in errors. For practical users, as long as the task allows for verifying answers afterward, a model that answers correctly nearly six times out of ten and fabricates only a tenth of the time is generally more useful than one that refuses to say anything. What you fear isn't a model that makes mistakes; it's a model that errs without you knowing. Put these numbers together, and Carbon's profile is: bold, higher accuracy, higher fabrication rate, but its absolute fabrication rate is still far below same-priced rivals.
How the $7.76-Per-Task Math Works
The evaluators gave a very practical metric: single-task cost. At standard pricing, Carbon runs one Intelligence Index task for an average of $7.76, about 23% cheaper than Opus 5.5's $10.04, but still about 1.2 times GPT-6 Astra's $6.25. In efficiency, Carbon outputs an average of 123k tokens per task, 29% fewer than Opus 5.5's 173k, indicating it tends to use shorter reasoning chains to reach answers—fewer words but the job gets done. Fewer output tokens brings a second benefit: shorter reasoning chains also mean less waiting time.
Breaking down the bill reveals more. Carbon's official pricing is $4 per million input tokens and $20 per million output tokens, exactly the same as Argon. Using $7.76 and 123k output tokens to reverse-engineer, an average task consumes around 1.3 million input tokens—a typical agent-load profile where prompts and tool returns dominate. Cached input costs 95% less, at $0.2 per million, so contexts with lots of repetition see costs drop sharply.
Running your own load through the numbers beats reading any evaluation. The script below computes single-task cost at official prices and can be run directly.
def task_cost(input_tokens, output_tokens, cached_tokens=0):
# Carbon official prices: input $4, output $20 per 1M tokens
# Cached input 95% off, or $0.2
normal = input_tokens - cached_tokens
return (normal * 4 + cached_tokens * 0.2 + output_tokens * 20) / 1_000_000
# Artificial Analysis measured: average 123k output tokens per Intelligence Index task
# Reverse-engineered: ~1.3M input tokens at official $7.76
plain = task_cost(1_300_000, 123_000)
cached = task_cost(1_300_000, 123_000, cached_tokens=1_000_000)
print(f"No cache ~${plain:.2f}, with 1M cached ~${cached:.2f}")
The result is about $7.66 without caching, and about $3.86 after hitting 1 million cached tokens—nearly halved. Google also offers a 50% launch discount with no announced deadline, so actual costs during the launch period can be cut in half again. The bill's conclusion is plain: whether it's expensive doesn't depend on leaderboard rank; it depends on how much of your input can hit the cache.
Same Price, 9 Points Higher, Argon's Position Looks Awkward
Carbon and Argon are priced identically—both at $4 per million input and $20 per million output—but the Intelligence Index rose from 50 to 59. A 9-point increase at the same price leaves users who just got Argon access in a delicate spot. The knowledge-work comparison is even more direct: on AA-Briefcase, Carbon scored 1609 Elo, 126 points above Argon's 1483. AA-Briefcase tests a model's ability to handle real office tasks and ranks with Elo ratings. Among its sub-items, presentation creation jumped 293 points to 1598, and analysis quality rose 120 points to 1671—both exactly the kind of tasks knowledge workers touch every week.
But don't rush to call it a blowout. On knowledge work, Carbon still isn't first. Three rivals sit ahead: Grok 4.7 at 1638, Muse Wonder at 1659, and Kimi K3.1 Pro at 1678. Kimi K3.1 Pro beats Carbon by nearly 70 Elo points, a non-trivial gap. A domestic model besting Google's new flagship on the knowledge-work leaderboard would have been hard to imagine a year ago. It shows the office-automation race is far from settled, and the next six months will likely reshuffle this board again.
Putting the four key numbers side by side makes the picture clearer.
| Model | Intelligence Index v5 | Coding Agent Index | Hallucination Rate | Single-Task Cost |
|---|---|---|---|---|
| Gemini 4 Carbon (high) | 59, 1st | 59, 2nd | 11% | $7.76 |
| Claude Opus 5.5 (max) | Not top | 62, 1st | 50% | $10.04 |
| GPT-6 Astra (max) | Not top | Not in top 2 | 50% | $6.25 |
| Gemini 4 Argon (high) | 50 | Below Carbon | 4% | $7.68 |
A clear division of labor emerges: choose Claude for coding, Carbon for overall intelligence, the older Argon for extremely low hallucinations, and GPT-6 Astra for budget-sensitive needs. There is no all-around champion—only ordering by scenario. That's the most useful way to read this leaderboard for the average developer. Whoever tells you one model crushes everything is advertising for the vendor. The results of your own load tests will always outrank any leaderboard.
The Release Rhythm Itself Becomes a Target of Criticism
Among the 200-plus Hacker News comments, the hottest thread wasn't the scores but Google's release rhythm. Like Argon, Carbon was announced, topped the chart immediately, and offered no opening date. One user said flatly: other companies let you use it within days of announcement, but Google plays the "sometime later" routine. With ability improving this fast, this kind of artificial scarcity doesn't hold people—they click in, see they can't use it, and leave. Veteran users also dredged up old grievances, saying the Gemini team has always excelled at releasing pretty benchmark charts and then disappearing on actual usability; when you finally get access, it's often expensive and hard to use.
More pointed speculation targets the launch timing. Some comments traced the timeline: Argon's announcement coincided with Google's internal performance review season, and Carbon's announcement hit right before quarterly earnings. A chorus agreed, saying these launches target not developers but review committees and shareholders. Others defended from an engineering angle, arguing that announcing first and slowly scaling out is standard practice in large-model safety processes, citing Anthropic's Mythos as an example that was also announced months in advance. Both interpretations have merit, but they burn the same resource: developers' trust budget for Google announcements.
As a user, you also got a concrete bitter pill. A Pro subscriber said Argon's quota is so low that five messages trigger a four-hour cooldown, and the web interface once hallucinated content based on file names. Benchmark scores are run in a laboratory; these complaints are worn out in production. Listen to both sides so your expectations for the new model don't get distorted.
What to Do Before It Opens
Eliminate a misconception first: staring at the leaderboard waiting for open access is pointless. The Intelligence Index changes weekly, and by the time you get onto the waitlist for Carbon trial access, the leaderboard may already have a new name on top. A more reliable move is to audit your own load structure right now and see the ratio of input to output tokens. Carbon's pricing structure dictates that workloads with heavy inputs and high cache hits are more cost-effective; output-heavy, long-reasoning loads will see their advantage shrink. Along the way, organize fixed content like system prompts into reusable prefixes—every percentage point of cache hit rate is a direct discount. Get that groundwork clear now, and on the day it opens you'll know whether to migrate.
If you already have Argon access, run a targeted self-built evaluation. Pick twenty knowledge questions from your production environment where real errors occurred, feed them separately to Argon and the model you currently use, and record both the answer accuracy and fabrication rates, keeping the questions and results on file. Argon's 4% hallucination rate, bought through refusal habits, may actually be an advantage in scenarios that require answer verification. When Carbon opens, run the same set again. How the 4%-to-11% hallucination change lands on your data is closer to the truth than any public leaderboard.
For those who need to order right now, the picture is clear: continue with Claude for coding-heavy loads, consider GPT-6 Astra's $6.25 unit price for general tasks, and look at Kimi K3.1 Pro for knowledge-heavy work. The biggest significance of Carbon topping the chart isn't to make anyone switch models immediately—it's proof that Google pinned the price in place while pushing the score up 9 points. That price anchor will force all rivals to follow suit. When the earnings-season buzz fades, every payer benefits.