Bug Hunt Bench blind-tests frontier coding models on 105 planted real-repo bugs
PawelHuryn · x · 2026-09-03
The author responds that models including GLM and DeepSeek are already listed on his Bug Hunt Bench leaderboard, he just didn't export every view as screenshots.
The benchmark (bughunt.productcompass.pm) blind-tests frontier coding models on 105 planted bugs across real repos, one prompt per repo. The board supports score-vs-cost, score-vs-time and coverage views; run costs span roughly 200x, hence the log-scale cost axis.
Related event: Bug Hunt Bench: GPT-6 Astra tops 105 real-bug test at 48/105(11 posts)→
More from coding & agent
- Skills vs MCP vs RAG vs Memory: the four knowledge layers production agents need — MaryamMiradi · 2026-10-21
- When customer service bans AI agents, one goes incognito to fight Xfinity — MoonL88537 · 2026-10-21
- Every.to publishes raw interviews so readers can agent-check quotes in its Codex story — every · 2026-10-21
- One-prompt Blender: Voyager + Claude Opus 5.5 auto-connects to Blender and Unity — ravisparikh · 2026-10-21
- Aviation tracks every part's provenance — agents can reason over that graph — colinmcnamara · 2026-10-21
- Claude Code directed a full music video: 6.5 hours, 765M input tokens — colinmcnamara · 2026-10-21