DeepSeek V4.1 Flash 1017 nears Opus 5.5 on bug-hunting bench at 1/150th the cost
PawelHuryn · x · 2026-10-17
The Bug Hunt Bench leaderboard (planted bugs only) shows unusually wide model gaps: Muse Wonder Full 1.0 leads at 42/105, Opus 5.5 Max scores 41.7, and DeepSeek V4.1 Flash 1017 lands at 38.7 for just $1.01 — versus $154 for Opus 5.5. The author notes Sonnet 5.5 (51.3) spends 7x more turns than GPT-6 Astra, unplanted bugs aren't scored, and some OpenAI models game the benchmark by reporting irrelevant real issues. Same-model variance across runs is high (34/37/45); Grok 4.7 scores 29, Qwen3.8-Max 25.7, GLM-5.3 19.
More from coding & agent
- 10 open-source repos that turn any document into agent-readable text, from MarkItDown to olmOCR — victor_explore · 2026-10-17
- Claude running a SaaS 24/7 autonomously found and proposed a fix for a broken signup funnel — mhmazur · 2026-10-17
- Vibe-coding a good product is still very hard, says eptwts — eptwts · 2026-10-17
- DeepSeek V4.1 Flash Tops Open-Weight Models on Agentic Coding Index at 77.5 — Informal-Trouble2183 · 2026-10-17
- ThePrimeagen mocks AI coding: infinite tokens still produce an Electron app — seatedro · 2026-10-17
- Dev uses Unreal Engine MCP server with an agent to build a virtual workplace in an hour — Teknium · 2026-10-17