Bug Hunt Bench: New Blind-Graded Benchmark Tests Frontier Models on 105 Real Bugs
PawelHuryn · x · 2026-09-29
Pawel Huryn launched Bug Hunt Bench, a blind-graded benchmark testing frontier coding models on 105 real, re-planted bugs from real repos that frontier models missed in early 2026. Judges come from different model families, only fully fixed bugs score, and the answer key stays private to prevent gaming. The leaderboard also plots score against cost — runs span roughly 200x in cost — with full run notes on GitHub.
More from Models
- Tencent Hy4 flash and InternLM Sage-S2.1-Flash both released, tool-calling tests requested — MajesticPuzzler · 2026-11-16
- User Shares Coral Reef Creation Labeled 'Qwen 4 Max (xhigh)', an Unannounced Model Name — intellectronica · 2026-11-16
- Stealth Model 'silicon-2' Surfaces in Devin, Suspected to Be Anthropic's Next Model — kimmonismus · 2026-11-16
- Google hides Gemini model label in AI Mode, rolls Gemini 4 Carbon to Pro subscribers — gaganghotra_ · 2026-11-16
- Open-weight Karpo-Decision-31B tops Decision Index at 73.9, one forward pass per answer — CodeByPoonam · 2026-11-16
- AI daily: DeepSeek rate-limits chats, Qwen haggling spurs shop warnings, Doubao workspace launch — 快鲤鱼 · 2026-11-16