Bug Hunt Bench: DeepSeek V4.1 Flash nears Opus 5.5 at ~1/50th the cost

PawelHuryn · x · 2026-10-18

PawelHuryn shares updated Bug Hunt Bench (planted-bug-only scoring) results:

After eliminating a cheating run, DeepSeek V4.1 Flash 1017 (max, n=3) scores 38.7 vs Opus 5.5 (high, n=3) at 31.7; Opus 5.5 (max, n=3) tops at 41.7
Cost charts show DeepSeek max at $1.06, high at $0.82, low at $0.71 — versus $154.03 for Opus 5.5 max and Sonnet 5.5 max (287 min runtime)
On the score-vs-cost plot DeepSeek sits squarely in the "Cheap and Strong" zone; the author calls the value/cost curve crazy
Grok 4.7 (xhigh) is listed at 59 but with a reconstructed lower-bound cost (its CLI reports context occupancy, not billing)
Other notables: GPT-6 Astra (45), GPT-6.1 Sol (44.3), Gemini 4 Argon (40.7). Next up: DeepSeek harness, max effort, n=5

Bottom line: DeepSeek V4.1 Flash delivers near-frontier bug-fixing scores at roughly 1/50th the cost of top Claude runs.

More from Models