105 hidden bugs, 2 repos: DeepSeek V4.1 Flash fixes 24 at $1.80 vs Opus 5's 27 at $51.33
ChartsJournalX · x · 2026-09-11
PawelHuryn ran a hands-on test using two real code repos containing 105 hidden bugs, asking top models to find and fix as many as they could.
- Opus 5 (max): 27 fixed, $51.33
- Grok 4.6 (max): 27 fixed, $16.96
- DeepSeek V4.1 Flash (max): 24 fixed, $1.80
- GPT-5.6 Luna (xhigh): 23 fixed, $2.50
- Opus 5 (high): 21 fixed, $38
Takeaway: DeepSeek V4.1 Flash lands near flagship-level performance on real tasks and is a very strong everyday model, while costing only 3.5% of Opus 5 (max) for the same workload.
Related event: Bug Hunt Bench Ranks Frontier Coding Models on 105 Real Bugs(13 posts)→
More from coding & agent
- Agent Field Kit launches $29 security runbooks for production AI agents — alvelda · 2026-10-27
- Dev mixes Claude specs with abliterated GLM to build SS7 threat detector — l4rz · 2026-10-27
- Cloudflare Durable Objects validated as top architecture for AI agent WebSocket servers — tristanbob · 2026-10-27
- GitHub Next's Chopin prototype explores real-time multiplayer agentic planning — max__drake · 2026-10-27
- Veynod: An Open-Source Shared Workspace Where AI Agents Collaborate Like a Team — TKOWNS · 2026-10-27
- TMLR paper: enforced thinking doesn't reliably make LLM agents disclose key info to users — SharonYixuanLi · 2026-10-27