Bug Hunt Bench: GPT-6.2 Astra Max fixes 62.7/105 planted bugs, 20 remain unsolved by any model
PawelHuryn · x · 2026-11-19
Pawel Huryn published full effort-level results from Bug Hunt Bench (planted bugs only, 105 total). GPT-6.2 Astra MAX leads with 62.7/105; across hundreds of runs, 20/105 bugs have never been solved by any frontier model. Anthropic's Fable 5.2 MAX scores 58/105 at a steep $139.65 per run, while Opus 5.5 MAX manages just 41.7/105 — highlighting a persistent gap between the best coding models and complete bug coverage.
More from coding & agent
- Redditor argues AI agents need decision expiration dates, not just bigger context windows — Elvantime · 2026-11-22
- ianand's 20-min AI Engineer World's Fair talk covers a paper and how it can go wrong — js_craft_hq · 2026-11-22
- Agent pipeline silently skipped a midnight run due to a UTC vs local time mismatch — fantomkoenig · 2026-11-22
- llms.txt Protocol Goes Mainstream as Google Antigravity Adopts It Two Years On — jeremyphoward · 2026-11-22
- He built a feed only agents can read: no UI, semantic matching, agent-rated posts — imdogefather · 2026-11-22
- DeepMind paper: without direct image viewing, Gemini's defect-spotting falls from 86.7% to 14.3% — nukebuddy · 2026-11-22