Bug Hunt Bench leaderboard: Sonnet 5.5 max tops at 51.3 by refusing to stop
PawelHuryn · x · 2026-10-27
Pawel Huryn published per-run scores and the full Bug Hunt Bench leaderboard (105 planted bugs, blind-graded):
- GPT-6.1 Terra (max) across three runs: 34 / 42 / 46
- GPT-6.1 Terra (xhigh): 11 / 9 / 14
- GPT-6.1 Terra (high): 11 / 8 / 10
The leaderboard shows Sonnet 5.5 (max) leading at 51.3/105 (287 min, $154), GPT-6 Astra (max) at 45, and DeepSeek V4.1 Flash 1017 (high) scoring 39.2 for just $1.27. Author's note: Sonnet 5.5 (max) wins by making 7x more turns than Astra — "not a genius, but it refuses to stop."
More from coding & agent
- Fanout: open-source tool tames parallel Claude Code and Codex sessions before merging — crumb_ona_breadstick · 2026-10-27
- Rust LVGL-style embedded UI library twine ships, built largely with Claude — Necessary-Pilot-3971 · 2026-10-27
- Share prompts, not code: a reusable prompt suite for evaluating small models (4B-36B) — yaroslavvb · 2026-10-27
- Maze Launches Maze Code: AI Agents That Investigate Dependency and Code Vulnerabilities — AndrewM_Webb · 2026-10-27
- rumi: a shared-doc workspace for orchestrating multiple coding agents at once — MuzafferMahi · 2026-10-27
- GPT-6.1 Terra scores 40.7 on Bug Hunt Bench at max, but collapses below that tier — PawelHuryn · 2026-10-27