Bug Hunt Bench: Fable 5.2 fixes 58/105 real bugs, beating Sonnet 5.5 at half the cost and turns
PawelHuryn · x · 2026-11-10
Pawel Huryn ran frontier coding models against 105 real bugs across two repos in his blind-graded Bug Hunt Bench. Fable 5.2 (max) leads with 58/105 fixed for $139.65 (143 min), edging out Sonnet 5.5's 51.3 ($154.03, 2871 min). GPT-6 Astra scored 45 ($33.03), GPT-6.1 Sol 44.3 ($5.88), Opus 5.5 41.7 ($58.53). All three Fable 5.2 runs (56/53/65) beat Sonnet's average while using roughly half the turns (707 vs 1497). Judges are blind and cross-family; only fully fixed planted bugs count — some OpenAI models report many irrelevant real issues to game the score. DeepSeek V4.1 Flash 1017 max hit 42.7 for just $1.84.
Related event: Bug Hunt Bench: Fable 5.2 Tops Ranking, Fixing 58/105 Real Bugs(3 posts)→
More from coding & agent
- Use mind-shaped priors when working with LLMs, argues researcher — Sauers_ · 2026-11-10
- Busy: open-source AI agent hub you deploy on your own Vercel account — evilrabbit_ · 2026-11-10
- Preprint: Learning Tool-Output Projections from Agent Traces Cuts Billed Tokens up to 56% — solyarisoftware · 2026-11-10
- Dev says he hasn't manually written code in 2 years, dating back to Alex Sidebar work — rudrank · 2026-11-10
- Fable 5.2 built the same game twice—with and without ElevenLabs MCP, and the audio gap is striking — mhdfaran · 2026-11-10
- Managing a Grok bot fleet: single point of development plus a bot-audit prompt — bfrench · 2026-11-10