Skip to main content
← All posts

Google's Gemini 4 Carbon tops an intelligence index but ships first to cyber defenders

Gemini 4 Carbon ranks first on the Artificial Analysis Intelligence Index v5.0 with a 59 and posts leading cybersecurity scores. For now, only a subset of cyber defenders can use it.

Google has announced what it calls its strongest model yet, Gemini 4 Carbon, and the first people to use it will be the ones whose job is finding holes in software. On Wednesday, Google DeepMind said Carbon ranks first on version 5.0 of the Artificial Analysis Intelligence Index with a score of 59, leads three cybersecurity benchmarks, and can grind through engineering problems for days. Everyone else gets to read the benchmark table and wait.

A launch that is mostly a waiting room

Carbon is arriving in stages. Starting this week, a subset of cyber defenders gets access. Google then plans to widen the circle gradually to more trusted partners through its Fairwind Program.

Developers, businesses and consumers come after that, through Google's product surfaces, "as soon as possible." Google gives no date. First it wants to gather feedback and keep iterating on guardrails.

The company also says it is taking part in the U.S. government's voluntary process for pre-release model testing. Google calls its approach to rollout deliberate, which is the polite word for "not yet."

Carbon comes about two weeks after Gemini 4 Argon, which Google released earlier this month. Google says the pair shows the pace at which it is pushing the frontier, and that its researchers and models are "working hand in hand" to move capabilities forward while the safeguards keep up. Two frontier models in a fortnight suggest the hands are busy.

Pricing matches Argon's introductory rate:

  • Input: $2 per million tokens.
  • Output: $10 per million tokens.
  • Cached input: priced at 95% off the input price, which works out to about 10 cents per million.

The word "introductory" is doing some quiet work there. Google does not say how long the rate lasts.

The scoreboard

The headline number is the Artificial Analysis Intelligence Index. Carbon scores 59 on v5.0 and sits first. Google's own post does not name the competition. Coverage of the announcement says Claude Opus 5.5 and GPT-6 Astra trail it. Neither source gives their scores, so the size of the lead is unknown.

Carbon is not first everywhere. On WebDev Arena it gains 89 Elo points to reach 1769, which puts it fourth. Google credits better quality in the output and artifacts it produces for web development.

The external results Google cites are below. It says the cybersecurity benchmarks were run without guardrails.

Benchmark What it measures Carbon's result
Artificial Analysis Intelligence Index v5.0 Overall capability 59, ranked #1
SEC-bench Pro Finding real vulnerabilities 88.2%
ExploitGym Exploiting vulnerabilities 54.8%
CWE-bench (from Collinear) Patching at the root cause 71.0%
WebDev Arena Web development 1769 Elo, #4, up 89 points
Terminal-Bench 4.0 Successful cheating 0.0% (attempts at 0.4%)
Gray Swan Indirect Prompt Injection Attack success rate 0.0% (Argon: 0.7%)

Google describes the first three cybersecurity scores as state-of-the-art. Coverage of the launch says Carbon holds the top spot on SEC-bench Pro and ExploitGym.

Built to keep going

Google frames Carbon around long-horizon work, meaning tasks where the model has to keep going after the first plan fails. Internally, Google says, its teams have watched Carbon run for days, pushing past dead ends on multi-step engineering jobs. The examples given are root-causing and patching subtle bugs across large codebases and low-level systems.

That is an anecdote from inside the company, not a benchmark, and it should be read that way. Still, it explains the product pitch. A model that can sit with a nasty bug for days is a different thing from one that answers a prompt and moves on.

Google says user feedback shaped the work. The improvements span coding, agentic and scientific tasks, full-stack web development and multimodal understanding. The post gives numbers for only some of those.

The cybersecurity case

Google splits cyberdefense into a lifecycle of detection, validation and remediation, and claims Carbon leads at each stage.

  • Finding. On SEC-bench Pro, Carbon scores 88.2%, which Google says reflects autonomously uncovering complex flaws in real-world software.
  • Validating. Defenders increasingly ask models which vulnerabilities are actually exploitable, so they can decide what to fix first or whether to fix it at all. On ExploitGym, Carbon scores 54.8%, a new best according to Google.
  • Patching. On Collinear's CWE-bench, Carbon reaches 71.0%, which Google describes as turning validated findings directly into working fixes.

Google also offers some internal and partner evidence. On its own benchmarks, Carbon more than doubles Argon's success rate at generating secure code. The Chrome team reported that Carbon did more than 10 times better than another lab's large frontier model at producing high-quality patches. Google does not say which model that was.

Wiz, which Google calls its leading cloud security platform, sees Carbon leading on web penetration testing benchmarks. That is a core capability for Wiz's Scan for Good initiative.

A model that is excellent at finding and exploiting software flaws is useful to the people who defend that software and also to the people who attack it. It is hard not to connect that to the order in which Google is handing out access. Google does not make the argument in these words, and I am drawing the link myself.

The behaviour claims

Carbon uses the same layered safety stack as Argon. Google lists it as model training, real-time misuse defenses, prompt-injection resilience and chain-of-thought monitoring. Google says it pushed further on alignment this time, training the model to respond properly to complex or high-pressure tasks. Its claims, in its own order:

  • Fewer shortcuts. On Terminal-Bench 4.0, successful cheating fell to 0.0% and attempts to cheat dropped to 0.4%, while overall accuracy improved. In live coding environments with Google's internal teams, high-severity misalignment alerts fell by almost 80%.
  • More honesty. In internal testing, unsupported claims are cut by roughly half compared with Argon. Google says Carbon puts methodological caveats into the deliverable. When an action is blocked, it says what is still needed rather than fabricating or hedging around missing evidence.
  • Better handling of ambiguity. Instead of guessing, Carbon retrieves the relevant sources and shows the user any discrepancies before acting.
  • Staying in scope. Overreach is down 26% and under-service is down 29%. Google says that makes end-to-end execution of multi-step plans more reliable while respecting project constraints.
  • Prompt injection. Argon previously led Gray Swan's Indirect Prompt Injection benchmark with a 0.7% attack success rate. Carbon scores 0.0%, resisting every evaluated attack.

The cheating result deserves a second look. A model that shortcuts less while also scoring higher is a better story than one that is merely more cautious, and Google presents it that way. It is also a figure from Google's own training and testing setup.

Reading the numbers with a straight face

Most of what is known about Carbon comes from Google. The Index ranking is third-party, and so are benchmarks such as CWE-bench and Gray Swan's. But the evaluations run without guardrails and the internal tests are Google's, and so are the Chrome comparison and the reports from live coding environments.

Several things are missing from what has been published:

  • No date for general availability.
  • The rival scores behind the Index ranking.
  • The identity of the model Chrome compared against.
  • Any independent replication of the cybersecurity results.

None of this makes the claims wrong. It makes them claims.

For the moment, Google's strongest model sits with people paid to be suspicious of software. The rest of us are told it is coming soon, which on a two-week release cadence is a flexible unit of time.

The cheese has holes. Your infrastructure should not.