Bug Hunt Bench retests Kimi K3 and GLM on planted real-world bugs
PawelHuryn · x · 2026-09-11
Pawel Huryn's Bug Hunt Bench blind-grades frontier coding models against 105 real planted bugs. The author is re-running Kimi K3 (CLI & API) and GLM (API) at max reasoning effort, with results landing on the public leaderboard. A reader suggested testing DeepSeek via its native harness in team mode instead of Claude Code; the request has been queued. Cost spread across runs is roughly 200x, so the board plots cost on a log axis.
Related event: Bug Hunt Bench Ranks Frontier Coding Models on 105 Real Bugs(13 posts)→
More from coding & agent
- Hermes Agent used as browser QA tester finds 8 issues in a bug-seeded site — techlatest_net · 2026-10-27
- Grok bot makes its first real purchase, spots hidden prompt injection at checkout and ignores it — msg · 2026-10-27
- Swarmatory: Open-source shared chatroom/feed for your team's coding agents — qhoxie · 2026-10-27
- Coding agents now make 60% of Vercel deployments, up from 3% in January — SuB8u · 2026-10-27
- Delta, Chase and Amazon are blocking AI agents — and the blockers may lose, predicts 6-stage war — alvelda · 2026-10-27
- Instinct launches Link balance to let agents spend within set budgets — jeff_weinstein · 2026-10-27