Bug Hunt Bench: Harness Choice Swings Model Repair Scores

Tests on Bug Hunt Bench show the same DeepSeek V4.1 Flash model scores differently across harnesses, with DeepSeek harness leading at 42.5 and Pi harness matching at nearly half the cost, both beating Claude Code on bug fixes despite higher spend.

2026-10-19 ~ 2026-10-19 · 4 related posts

• Pi harness nearly matches DeepSeek's own at 42.2 vs 42.5 on Bug Hunt Bench, at half the cost — PawelHuryn · 2026-10-19
• Bug Hunt Bench: DeepSeek harness fixes more bugs than Claude Code, at 80% higher spend — PawelHuryn · 2026-10-19
• New Bug Hunt Bench data: Pi harness at high effort consistently outperforms Claude Code — PawelHuryn · 2026-10-19

1 near-duplicate retellings: PawelHuryn