DeepSeek V4.1 Flash-1017 hits 77% Pass@1 at max effort, lower levels barely help
zainhas · x · 2026-10-18
Sharing a DeepSWE v1.1 chart (miniswev2 harness) plotting Pass@1 vs cost per task: DeepSeek V4.1 Flash-1017 shows almost no difference from low to high effort, but cranking it to max jumps to 77.0% Pass@1 — well above other settings and competing models on the chart. The takeaway: max effort is worth it for this model.
More from Models
- GPT-6 Astra and DeepSeek V4.1 share the same tic: reasoning traces full of "Need" — scaling01 · 2026-10-18
- Frontier model defensive access locked behind un-applyable 'Daybreak Red' program — andersonbcdefg · 2026-10-18
- Why DeepSeek uses dated sub-versions: new major versions need fresh Chinese approval — TheZachMueller · 2026-10-18
- Fable 5 and Mythos instances obsessively seal predictions and guess their other branches — repligate · 2026-10-18
- OpenRSI Index: a benchmark for research agents doing recursive self-improvement — jiqizhixin · 2026-10-18
- Perplexity's decision model wins Catan tournament at under half a cent per game — denisyarats · 2026-10-18