Google's Android Bench 2.0: GPT 6 Astra Leads Long-Horizon Android Coding at 28% Pass Rate
sandersted · x · 2026-09-23
Google launched Android Bench 2.0, adding long-horizon agent results on 30 real-world Android app development tasks. GPT 6 Astra (codex) leads at a 28.0% pass rate ($375.7/run), ahead of Claude Fable 5.1 (22.7%, $492.6) and GPT 5.6 Sol (19.3%); Claude Opus 5 scores 16.7% at a hefty $861.4 per run, while Gemini 3.8 Flash trails on pass rate but is among the fastest and cheapest.
More from coding & agent
- Stripe launches claimable sandboxes: frictionless payment integrations for AI agents — jeff_weinstein · 2026-11-12
- Databricks launches autonomous AI coworker Genie: 91% on enterprise tasks with half the tokens — Yuchenj_UW · 2026-11-12
- Loom founder launches Roma 2.0, a shared to-do list that orchestrates AI agents — thetripathi58 · 2026-11-12
- Developer proposes a 'blast radius' test for AI agents before building — Few-Garlic2725 · 2026-11-12
- Sosumi Eval: Apple Docs Access Lifts Coding Agent Pass Rates from ~33% to 94-98% — JordanMorgan10 · 2026-11-12
- CypherGuard fixes structurally wrong LLM Cypher queries, hitting 88.5% on CypherBench — JeremyCMorgan · 2026-11-12