New Bug Hunt Bench data: Pi harness at high effort consistently outperforms Claude Code
PawelHuryn · x · 2026-10-19
PawelHuryn's Bug Hunt Bench has a new result: "Pi + high effort" is ready, and together with the DeepSeek Harness it consistently outperforms Claude Code on bugs fixed. The benchmark runs the same DeepSeek V4.1 Flash 1017 model across three harnesses and three effort levels against 105 planted bugs, with diffs blind-judged by Grok and three answer-leaking runs voided.
At Max effort, DeepSeek Harness hits a median of 42 (mean 42.7) and Pi 42 (42.2) vs Claude Code's 38 (39.1); at High effort Pi's median of 40 also beats Claude Code's 38. Low-effort runs are next, and all scores are published on the benchmark site.
Related event: Same Model, Different Harness: Bug Hunt Bench Shows 3.6-Point Spread(5 posts)→
More from coding & agent
- Claude uses Hedra MCP to auto-refit a music video into 9:16 vertical — AIandDesign · 2026-10-19
- Claude writes 90% of Claude Code's code; merged PRs doubled while reverts fell 40% — PrajwalTomar_ · 2026-10-19
- ESP32 + Claude Code tutorial: firmware, server setup and MCP integration in 2 minutes — alexcovo_eth · 2026-10-19
- 4 Gemini 4 Argon agents team up for 27 hours to beat Minecraft's Ender Dragon — robertarail · 2026-10-19
- Designing a non-bugged activation-read probe: numerics must not shift under the same prompt — brandon_xyzw · 2026-10-19
- OpenCode 2.0 ships: major release for the open-source terminal coding agent — RelevantEmergency707 · 2026-10-19