One code-review skill helps GPT-5.6, Grok 4.6, Opus 5 fix 4-13.5 more bugs
PawelHuryn · x · 2026-10-17
Pawel Huryn published results from his Bug Hunt Bench: 105 planted bugs across two real codebases, graded blind, same model/effort/judge — the only change was adding his code-review skill (from the pm-ai-shipping collection) to the prompt, which makes the model try to refute every finding before reporting it.
Results (bugs fixed /105):
- GPT-5.6 Sol: 43.5 → 57 (+13.5); 253→454 min; $95.35→$143.39
- Grok 4.6: 28.7 → 34 (+5.3); 43→62 min
- Opus 5: 27 → 31 (+4); 60→117 min; $51.33→$88.46
Key takeaways:
- Every model improved; the strongest model gained the most
- The cost: 1.5-1.7x money and 1.4-2x time
- Single run per setup; repeat runs differ by a median 5/105 (up to 17), so read the direction, not the exact gain
- No subagents — one model, one context, skill pasted into the prompt
The skill doesn't name specific bugs; it focuses attention on bug classes models commonly miss.
Related event: Pawel Huryn Open-Sources pm-ai-shipping, Boosting AI Code Review(2 posts)→
More from coding & agent
- 48 hours of chaos: PHP, GitLab CVSS 10.0, Chrome's 353 fixes — audit your stack now — reach_vb · 2026-10-23
- Neon ships official AI Gateway adapter for TanStack AI with single-credential multi-model access — matei_zaharia · 2026-10-23
- Modal open-sources Spindle, a Tinker-compatible training server you pay for in GPU time — andersonbcdefg · 2026-10-23
- Reflection AI's Beam agent auto-builds a Gemma-4 fine-tuning notebook, boosting accuracy 66.5% — danielhanchen · 2026-10-23
- Cua Spaces SDK lets Hermes and other agents run in sandboxes on your own hardware — Teknium · 2026-10-23
- PHP hit by two unauthenticated RCE flaws; ProjectDiscovery launches free internet-wide alert service — cyb3rops · 2026-10-23