Bug Hunt Bench: 105 real bugs show GPT-6 scoring 3-5x higher than rival models
PawelHuryn · x · 2026-10-27
Developer Pawel Huryn released Bug Hunt Bench, a blind-graded benchmark of 105 real bugs extracted from two production repos. Frontier models including GPT-6, Claude, Grok, Gemini and DeepSeek hunt and fix bugs in their own CLIs; the leaderboard and all receipts are public on GitHub.
Per-run scores across three runs:
- max: 34 / 42 / 46
- xhigh: 11 / 9 / 14
- high: 11 / 8 / 10
- medium: 8 / 6 / 8
- low: 7 / 5 / 6
The max tier outscores other tiers by 3-5x. Full source data is open for anyone to reproduce with their own agent.
More from coding & agent
- Compound Engineering 3.33.0 ships PR workflow upgrades to curb scope-creeping agents — kieranklaassen · 2026-10-27
- Lovable invokes 753.6B Cloudflare Workers monthly, migrated from Next.js to TanStack Start — ritakozlov · 2026-10-27
- Harness Lens now indexes 30 open source coding harnesses and accepts new submissions — yb2698 · 2026-10-27
- Prompting agents at the grammar level: pattern language as interface design — DanGrover · 2026-10-27
- Microsoft Foundry hosted agents get mailboxes, identity and org-chart presence — marlene_zw · 2026-10-27
- 36 blind AI judges pick a solo-session agent setup over Superpowers 36-0 — Kai_ThoughtArchitect · 2026-10-27