Bug Hunt Bench: DeepSeek harness fixes more bugs than Claude Code, at 80% higher spend
PawelHuryn · x · 2026-10-19
PawelHuryn published the full per-run scores of Bug Hunt Bench: the same DeepSeek V4.1 Flash 1017 model run across three harnesses (Claude Code, DeepSeek Harness, Pi) and three effort levels, fixing planted bugs out of 105, blind-judged by Grok, with three runs voided for reading answers outside their folder.
Key numbers: at Max effort, DeepSeek Harness hits a median of 42 (mean 42.7) vs Claude Code's 38 (39.1); Pi also reaches a median of 42. But as discussants note, that's 2-3 extra points for roughly 80% more spend — a tough trade outside of evals.
Related event: Same Model, Different Harness: Bug Hunt Bench Shows 3.6-Point Spread(5 posts)→
More from coding & agent
- Claude uses Hedra MCP to auto-refit a music video into 9:16 vertical — AIandDesign · 2026-10-19
- Claude writes 90% of Claude Code's code; merged PRs doubled while reverts fell 40% — PrajwalTomar_ · 2026-10-19
- ESP32 + Claude Code tutorial: firmware, server setup and MCP integration in 2 minutes — alexcovo_eth · 2026-10-19
- 4 Gemini 4 Argon agents team up for 27 hours to beat Minecraft's Ender Dragon — robertarail · 2026-10-19
- Designing a non-bugged activation-read probe: numerics must not shift under the same prompt — brandon_xyzw · 2026-10-19
- OpenCode 2.0 ships: major release for the open-source terminal coding agent — RelevantEmergency707 · 2026-10-19