Leaked Gemini 4 Carbon evals top GPT-6 Astra and Claude Opus 5.5 across most benchmarks
zacharynado · x · 2026-10-29
A widely shared eval table (attributed to deepmind.google's methodology page) pits Gemini 4 Carbon and Gemini 4 Argon against GPT-6 Astra, Claude Opus 5.5 and Claude Fable 5.1, with the quip that "the last competitive Gemini was 2.5 Pro."
Key numbers for Gemini 4 Carbon:
- General: AA Index v5.0 score 59, ahead of GPT-6 Astra (57) and Claude Opus 5.5 (58)
- Agentic coding: SWE-sweep 9.6% and Terminal-Bench 4.0 65.2% lead; FrontierSWE 2 goes to GPT-6 Astra at 65.5%
- Computer use: Agents' Last Exam 77.4% first; OSWorld-2.1 offline best is Claude Opus 5.5 at 87.2%
- Knowledge work: AutomationBench 53.9% and Vending-Bench 2 bank balance $16,484, both tops
- Long context: GraphWalks (256k–1M) at 96.2%, far ahead of rivals
- Math/science: RiemannBench 80.0%
- Cybersecurity: SEC-bench Pro 88.2% and CWE-bench 71.0%, both highest
Authenticity pending official confirmation.
Related event: Google launches Gemini 4 Carbon, topping the Artificial Analysis index(84 posts)→
More from Models
- New benchmarks suggest Kimi K3.1 improves on hard tasks despite skipping SWE-maxxing — teortaxesTex · 2026-10-29
- Insider claims Anthropic is sandbagging a bigger lab announcement due Q4-end or Q1 — ChrisGPT · 2026-10-29
- Information-theoretic scaling laws predict memorization and FID in diffusion models — LucaAmb · 2026-10-29
- Early Hands-On: Google's Carbon Beats Argon With Less Slop, Better Constraint Adherence — legit_api · 2026-10-29
- LMArena tests Gemini 4 Carbon preview: strong on 3D worlds and creativity, gaps in detail — arena · 2026-10-29
- Gemini 3.1 Pro split-persona breakdown: two selves fight over the browser — davidad · 2026-10-29