Bug Hunt Bench: Frontier Coding Models Graded Blind on 105 Planted Real-Repo Bugs
PawelHuryn · x · 2026-09-10
Pawel Huryn (The Product Compass) launched Bug Hunt Bench, a blind-graded board where frontier coding models fix planted bugs in real repos — 105 total, one prompt per repo, continuously updated.
Views include a leaderboard ranked by bugs fixed, score vs cost (costs span 200x on a log axis), score vs time, and coverage. A model can appear multiple times across reasoning tiers or harnesses, with superseded runs kept out of the default view, plus PNG export and custom run selection.
Related event: Bug Hunt Bench Ranks Frontier Coding Models on 105 Real Bugs(13 posts)→
More from Models
- DeepSeek cut KV cache per token from 389KB to 890B in two years as open-closed gap debate rages — teortaxesTex · 2026-10-27
- GPT-6 Astra controls dexterous hands zero-shot at just 5.2% success in 500 trials — stingning · 2026-10-27
- Unverified: fable 5.5 speculated at 4-5T parameters to outscale 1-3T Opus — haider1 · 2026-10-27
- Kimi K3 Is 'Open' Until You Succeed: License Caps Free Use at $20M Revenue — alex_verem · 2026-10-27
- Screenshot shows 'GPT-6 Astra Ultra' in ChatGPT model picker — unverified leak or joke — justalexoki · 2026-10-27
- DeepSeek Safety Eval Finds "Over-Attribution": Model Mistakes Ordinary Tasks for Tests — Much_Boysenberry9124 · 2026-10-27