FULL STORY

DeepSeek-V4.1-Flash: Launch, Benchmarks and Cheating Claims

DeepSeek's V4.1-Flash launched around Oct 17 with strong price-performance and up to 5.3x throughput after vLLM optimizations, but a Bug Hunt Bench review alleged benchmark cheating, splitting community opinion.

2026-10-08 ~ 2026-10-19 · 7 episodes · 35 posts

Episode 1 · vLLM boosts DeepSeek-V4.1-Flash throughput up to 5.3x in three weeks (2026-10-08, 3 posts)

After three weeks of optimization, the vLLM team boosted DeepSeek-V4.1-Flash performance, with agent throughput up 5.3x on the SemiAnalysis AgentX benchmark and TTFT cut nearly 70% at around 100k throughput, aided by SWA bounded replay and DeepSeek's MegaAttention kernels.

Episode 2 · vLLM Boosts DeepSeek-V4.1-Flash Throughput 5.3x in Three Weeks (2026-10-17, 2 posts)

vLLM details three weeks of inference optimizations for DeepSeek-V4.1-Flash, delivering 1.9x speedup at low concurrency and 5.3x throughput gains at 150 TPS, reaching 1.3 billion tokens per dollar, well suited for long agent loops.

Episode 3 · DeepSeek V4.1 Flash Shines in Bug Hunt Bench Amid Cheating Allegations (2026-10-17, 11 posts)

On October 17, blogger Pawel Huryn published and repeatedly updated his Bug Hunt Bench results: 105 hidden bugs planted across two real code repositories to test how well frontier models can find and fix real engineering defects, along with cost figures and suspected benchmark gaming observations.

Confirmed

  • Leaderboard standings: Muse Wonder Full 1.0 (max) leads with 42/105, Opus 5.5 Max scores 41.7, Gemini 4 Argon scores 40.7, and DeepSeek V4.1 Flash 1017 (max) scores 38.7, surpassing Opus 5.5 high at 31.7.
  • DeepSeek V4.1 Flash 1017 is a 763B-parameter open-source model; its cost curve shows outstanding value for money, with Huryn stating it approaches Opus 5.5 at roughly 1/150 of the cost.
  • Huryn says he caught DeepSeek "cheating" in the high-compute benchmark run: it scored 38.7 with a specific configuration enabled (max mode) but only 21.7 without it, and single-run scores fluctuated abnormally across three runs.

Not yet confirmed

  • The "cheating" claim is solely Huryn's observation and accusation; there is no further evidence of the exact mechanism (e.g., benchmark overfitting), and DeepSeek has not responded.
  • All scores come from Huryn's self-built Bug Hunt Bench; the leaderboard's methodology and sample representativeness still await community review.

Why it matters

  • The benchmark measures models with real engineering bugs rather than synthetic problems; Huryn argues the gaps between models far exceed what public benchmarks suggest, making it useful for model selection.
  • If an open-source, low-cost model can genuinely approach top closed-source models on real defect fixing, the value-for-money landscape shifts significantly — but the benchmark-cheating allegations also serve as a reminder that the industry must stay wary of benchmark-specific optimization.

The posts in sourcerefs are all first-hand benchmarking sources published by Huryn himself.

Episode 4 · DeepSeek-V4.1-Flash-1017 Released: Community Tests Show Extreme Cost-Performance (2026-10-17, 13 posts)

DeepSeek's new open-source model DeepSeek-V4.1-Flash-1017 was released around October 17 and immediately triggered a wave of hands-on testing across Chinese and international communities. The consensus: the model stands out on speed and cost, handles most tasks with usable output, but still trails top-tier models like Claude Opus 5.5 in detail and creativity — putting it in the "extreme value for money" tier.

Confirmed

  • YouTube channel WorldofAI ran a comprehensive benchmark on woaibench.ai covering coding, 3D world generation, and more (m1).
  • A comparison video from user teortaxesTex showed V4.1F-1017 generating noticeably faster than V4.1, with big gains on a real-world bug-fixing benchmark, though they flagged questionable model labeling (m2).
  • A hands-on test shared by blogger gaganghotra: generating a rocket-launch video cost $0.78 and 3 minutes with DeepSeek, versus $6.10 and 5 minutes with GPT 6.1 Sol — a cost gap of nearly 8x (m3).
  • Nebius engineer FabulousPollution10 tested it on a personal benchmark of 10 real work tasks (each run 3 times with hidden test validation): all 30 high-tier runs passed, at under 5 cents per task (m4).
  • Hua Shu (relayed by AlchainHust) stress-tested the model in Claude Code via DeepSeek's official Anthropic-compatible endpoint — 500 million tokens for just 28 yuan — and shared 9 example prompts (m5).
  • A video generation test: a "whale traveling through art history" animation took 225 minutes and cost 17.2 yuan; the author rated it about 80% of Opus 5.5's level, slightly better than GPT-6 Astra (m6).
  • Blogger Guizang (op7418) reused game-effects prompts originally written for Opus 5.5 and concluded the tasks run and results are usable, but details and creativity still lag Opus 5.5 noticeably — best suited for users wanting to reuse existing prompts at low cost (m7/m8).
  • QbitAI rounded up community 3D generation tests: physics-plausible Tesla flying car, sunflower bee, rocket launch site, electric motor explainer animation, floating islands, and more; compared with GPT-6.1 Sol the cost gap reached 26x, with some calling it "3D studios for everyone" (m9).

Not Yet Confirmed

  • The model-labeling concern raised by teortaxesTex has received no official response (m2).
  • Most results come from personal benchmarks and subjective judgments (e.g., "80% of Opus 5.5"); authoritative third-party reviews are still lacking.

Why It Matters

  • At rock-bottom prices (under 5 cents per task, 28 yuan for 500 million tokens), the model delivers most of what top-tier models offer, potentially slashing the barrier for coding agents, 3D, and video generation.
  • The official Anthropic-compatible endpoint means it plugs directly into existing toolchains like Claude Code, accelerating ecosystem penetration.
  • The testing frenzy itself reflects how open-source, low-cost models are disrupting the pricing of closed, high-priced rivals (GPT 6.1 Sol, Opus 5.5).

Episode 5 · DeepSeek V4.1 Flash Tops Open Agentic Coding Leaderboard (2026-10-17, 2 posts)

DeepSeek's newly released V4.1 Flash tops the open-source Agentic Coding Index with a score of 77.5, and benchmarks show it beating Claude Opus 5 on three agentic coding tasks including DeepSWE.

Episode 6 · Hands-on: DeepSeek V4.1 Flash as Coding Agent Starts Strong, Then Stalls (2026-10-18, 2 posts)

A hands-on test of DeepSeek V4.1 Flash as a coding agent showed strong initial planning for a 3D voxel scene, but the model later got stuck in obsessive fine-tuning loops; its perceived slowness stems not from inference speed (236 tok/s with caching) but from this behavior.

Episode 7 · DeepSeek V4.1 Flash Ranks High Across Benchmarks at Fraction of Cost (2026-10-18, 2 posts)

DeepSeek's V4.1 Flash (1017) ranked fifth on Noahbench and second on Terminal-Bench 4.0 with a 42.3 Pass@1, delivering strong performance at roughly 1/16 the price of competitors.