Bug Hunt Bench leaderboard live: 105 planted bugs, score vs cost across frontier models
PawelHuryn · x · 2026-09-24
Pawel Huryn published the full Bug Hunt Bench leaderboard: blind-graded runs against 105 planted bugs in real repos, one prompt per repo, with score-vs-cost (log axis, roughly 200x spread) and score-vs-time (linear, under 7x) views. Full run notes and caveats live on GitHub in results/run-notes.md. The board is maintained alongside his newsletter The Product Compass.
More from coding & agent
- Loom founder launches Roma 2.0, a shared to-do list that orchestrates AI agents — thetripathi58 · 2026-11-12
- Sosumi Eval: Apple Docs Access Lifts Coding Agent Pass Rates from ~33% to 94-98% — JordanMorgan10 · 2026-11-12
- CypherGuard fixes structurally wrong LLM Cypher queries, hitting 88.5% on CypherBench — JeremyCMorgan · 2026-11-12
- Building an End-to-End Hedge Fund in Python with Claude Code: A 5-Step Workflow — mdancho84 · 2026-11-12
- Jeff Dean: comfortable with engineers spending up to $10M/year on AI tools — deedydas · 2026-11-12
- Developer builds VladOS, a personal life dashboard, by just describing it to an AI agent — haltakov · 2026-11-12