Gemini 4 Carbon scores 57.3 on 105 real bugs at $31, undercutting rivals by 4x on cost
DynamicWebPaige · x · 2026-11-11
A hands-on test of Gemini 4 Carbon (high) across 2 repos and 105 replanted bugs puts it at 57.3 — just behind Fable 5.2's 58 — but at $31.32 versus Fable's $139.65. Sonnet 5.5 scored 51.3 ($154.03), GPT-6 Astra 45 ($33.03), GPT-6.1 Sol 44.3 ($5.88), and the previous Gemini 4 Argon 40.7 ($7.16). That's a +16.6-point jump over Argon at under a quarter of Fable 5.2's cost, suggesting Google's own strong benchmarks weren't inflated.
More from coding & agent
- Stripe launches claimable sandboxes: frictionless payment integrations for AI agents — jeff_weinstein · 2026-11-12
- Databricks launches autonomous AI coworker Genie: 91% on enterprise tasks with half the tokens — Yuchenj_UW · 2026-11-12
- Loom founder launches Roma 2.0, a shared to-do list that orchestrates AI agents — thetripathi58 · 2026-11-12
- Developer proposes a 'blast radius' test for AI agents before building — Few-Garlic2725 · 2026-11-12
- Sosumi Eval: Apple Docs Access Lifts Coding Agent Pass Rates from ~33% to 94-98% — JordanMorgan10 · 2026-11-12
- CypherGuard fixes structurally wrong LLM Cypher queries, hitting 88.5% on CypherBench — JeremyCMorgan · 2026-11-12