DeepSeek V4.1-Flash-1017 Tops Open-Weight Leaderboards as Benchmark Discrepancies Emerge

On October 17, DeepSeek released V4.1-Flash-1017, officially positioned as a "minor update" with unchanged pricing and MIT-licensed weights — but third-party testing suggests otherwise: the model has taken the top spot among open-source models on multiple third-party and Arena-style leaderboards while becoming the lowest-hallucination DeepSeek release to date, drawing wide attention and discussion.

Confirmed

• Prompt Engineering's hands-on testing found that thinking tokens dropped by up to 45% in this version; official benchmark numbers rose across the board, and the model claimed first place among open-source models on the Arena WebDev leaderboard.
• Per Zhidongxi, DeepSeek-V4.1-Flash-1017 (Max) topped the open-source category on CodeArena:WebDev with a score of 1670, surpassing Kimi K3 (Max) as well as closed-source models like Claude Opus 5 and Grok 4.7, at roughly 1/10 the cost of Kimi K3.
• Vals AI's Vals Index shows the model (10/17 version) leading open-weight models with 56.58% accuracy, at a per-task cost of only $0.64 and a latency of 36 minutes 33 seconds.
• elliotarledge tested it on KernelBench-CUDA: all 4 CUDA kernels the model hand-wrote compiled successfully on an RTX Pro 6000.
• teortaxesTex assessed it as DeepSeek's least hallucination-prone 2026 version; his earlier prediction had placed it 45th–46th on the Artificial Analysis Intelligence Index, and the actual result held surprises.
• Developer Ibragim Badertdinov ran a comparison across 10 real-world work tasks (each model run three times in the Pi tool): the model passed all 10 programming tasks, at roughly 40% of Claude Opus's per-task cost.

Unconfirmed

• teortaxesTex questioned how the model's costs were calculated on the Artificial Analysis leaderboard, arguing its claimed cost advantage involves methodology issues.
• teortaxesTex noted the version regressed in knowledge and long-context ability, Pareto-dominated by Sol — a capability trade-off rather than across-the-board progress.
• On the Artificial Analysis CritPt physics reasoning benchmark, DeepSeek 1017 (max) ranked seventh at 24%, the highest-placed DeepSeek model, ahead of Kimi k3, GLM-5 Preview, and others — but teortaxesTex relayed comments calling the result mediocre and the cost advantage questionable.

Why it matters

At an extremely low price point, this release simultaneously tops open-source leaderboards across coding, agent tasks, CUDA kernel writing, and more, while sharply cutting thinking token consumption — a shake-up for open-model cost-performance dynamics. Yet it also exposes trade-offs in knowledge, long-context, and physics reasoning, a reminder that "topping leaderboards" is not the same as well-rounded capability, and its cost-accounting controversy merits attention.

2026-10-19 ~ 2026-10-20 · 12 related posts

Full story(4 episodes)→

• Episode 1: vLLM boosts DeepSeek-V4.1-Flash throughput up to 5.3x in three weeks(2026-10-08, 3 posts)
• Episode 2: vLLM Boosts DeepSeek-V4.1-Flash Throughput 5.3x in Three Weeks(2026-10-17, 2 posts)
• Episode 3: Hands-on: DeepSeek V4.1 Flash as Coding Agent Starts Strong, Then Stalls(2026-10-18, 2 posts)
• Episode 4: DeepSeek V4.1-Flash-1017 Tops Open-Weight Leaderboards as Benchmark Discrepancies Emerge(2026-10-19, 12 posts)

• DeepSeek V4.1-Flash-1017 is its least hallucinatory model yet, but regresses on knowledge and LCR — teortaxesTex · 2026-10-19
• DeepSeek V4.1-Flash-1017 Is the Least Hallucinatory DeepSeek Model, but Pareto-Dominated by Sol — teortaxesTex · 2026-10-19
• DeepSeek 1017 Scores 24% on CritPt Physics Benchmark, Cost Savings Questioned — teortaxesTex · 2026-10-19
• [source] DeepSeek-V4.1-Flash-1017 scores 10+ points lower on third-party Terminal bench tests — DigiDecode_ · 2026-10-19
• DeepSeek V4.1 Flash 1017 tops Vals Index open-weight ranking at just $0.64 per task — scaling01 · 2026-10-19
• DeepSeek V4.1 Flash 1017 reportedly scores 42.3 on Terminal-Bench 4.0, beating GLM-5.3 but only with its own harness — teortaxesTex · 2026-10-19
• [source] DeepSeek-V4.1-Flash tops CodeArena open-source web dev leaderboard at ~1/10 Kimi K3 cost — 智东西 · 2026-10-19
• DeepSeek V4.1-Flash passes all 10 coding tasks at 40% of Opus cost, but runs 4x slower — 创业邦 · 2026-10-19
• [source] DeepSeek V4.1 Flash 1017 tested: up to 45% fewer thinking tokens, #1 open model on Arena WebDev — Prompt Engineering · 2026-10-19
• DeepSeek V4.1 Flash writes CUDA kernels, tops KernelBench decode at 8.12% roofline — scaling01 · 2026-10-19
• DeepSeek V4.1 Flash 1017 hits 45 on AA Intelligence Index, tying #2 open-weights — ArtificialAnlys · 2026-10-20
• DeepSeek V4.1 Flash 1017 scores 45 on AA Intelligence Index, ranks #2, runs at 225 tok/s — ArtificialAnlys · 2026-10-20