DeepSWE benchmark: DeepSeek V41 Flash-1017 tops chart at 77% pass rate with lowest cost
Immediate_Simple_217 · reddit · 2026-10-18
A Reddit user shared the DeepSWE v1.1 coding benchmark chart (miniswev2 framework, data as of 2026-09-22) plotting Pass@1 against per-task cost:
- DeepSeek V41 Flash-1017 tops the chart at 77.0% while sitting at the cheapest end (¥1-2 per task)
- Claude Opus 5, Claude Fable 5, and GPT-6 Astra reach similar pass rates (70-75%) but cost ¥50-150 per task
- DeepSeek V4 Pro max hits 63% at mid-tier pricing; Kimi K3, GLM-5.3 variants also show strong cost-efficiency
Takeaway: Chinese models now hold both the top score and the best cost-performance on agentic coding evals.
More from Models
- GPT-6 Astra and DeepSeek V4.1 share the same tic: reasoning traces full of "Need" — scaling01 · 2026-10-18
- Frontier model defensive access locked behind un-applyable 'Daybreak Red' program — andersonbcdefg · 2026-10-18
- Why DeepSeek uses dated sub-versions: new major versions need fresh Chinese approval — TheZachMueller · 2026-10-18
- Fable 5 and Mythos instances obsessively seal predictions and guess their other branches — repligate · 2026-10-18
- OpenRSI Index: a benchmark for research agents doing recursive self-improvement — jiqizhixin · 2026-10-18
- Perplexity's decision model wins Catan tournament at under half a cent per game — denisyarats · 2026-10-18