Bug Hunt Bench: blind-graded leaderboard pits frontier coding models against 105 real planted bugs
PawelHuryn · x · 2026-10-24
Pawel Huryn, author of The Product Compass, launched Bug Hunt Bench — a blind-graded leaderboard running frontier coding models against 105 real planted bugs, one prompt per repo. Results can be ranked by score, cost, time, or turns, with per-run notes and caveats published on GitHub (results/run-notes.md and runs.csv).
The board supports comparing reasoning tiers and harnesses, with turn counts computed uniformly from each run's own logs. Huryn's stated standard: hands-on, no hype, nothing that wasn't run first.
More from coding & agent
- Screenshot shows multi-agent AI team with a Chief of Staff that self-corrects — RachelVT42 · 2026-10-24
- This all-in-one ESP32 pocket dev board is programmable by Codex and fits on a keychain — thesephist · 2026-10-24
- Desktop ComfyUI was unusable on phones, so he built a free open-source mobile UI — Deneme123deneme · 2026-10-24
- Open-source coding agent Kvist tackles crash recovery between file writes and checkpoints — ChatGPT_Codex · 2026-10-24
- Security veteran: 0day patch windows are now hours, security debt interest just spiked — xeophon · 2026-10-24
- Mubit open-sources Banyan Sapling, a verification model with calibrated probabilities (Apache 2.0) — testingcatalog · 2026-10-24