FULL STORY

DeepSeek V4.1 Flash Tops Security Benchmarks

DeepSeek V4.1 Flash beat GPT-6 Sol on Vals AI's cybersecurity benchmark and rediscovered 87.5% of CVEs at ~$2.4 each, before topping Artificial Analysis' updated Cyber Index among open models.

2026-10-20 ~ 2026-10-25 · 3 episodes · 13 posts

Episode 1 · Vals AI cyber benchmark: DeepSeek V4.1 Flash beats GPT-6 Sol (2026-10-20, 5 posts)

Third-party evaluator Vals AI published the first findings from its cybersecurity benchmark under construction: on multi-step offensive security tasks in simulated enterprise networks, DeepSeek V4.1 Flash 1017 scored 16.3/21 average completions, beating GPT-6 Sol (tested via the Daybreak trusted access program) at 15.7/21, with Grok 4.7 lower at around 12. The result was widely shared and discussed in the security community.

Confirmed

  • Vals AI's benchmark is built on simulated enterprise networks and measures multi-step offensive cybersecurity capabilities
  • DeepSeek V4.1 Flash 1017 averaged 16.3/21, above GPT-6 Sol Daybreak's 15.7/21; Grok 4.7 scored around 12
  • These are preliminary first findings; the benchmark is still being developed

Why it matters

  • Security researcher Joshua Saxe argued that as of Q4 2026, open-weight models skilled at dual-use cybersecurity tasks are a net positive for defense, since defenders can use them to improve detection and response
  • Open-source models matching or exceeding top closed models could reshape the offense-defense landscape, giving defenders more accessible, locally deployable tools
  • Reshares by multiple independent accounts show strong community interest in open-vs-closed cyber capability comparisons

Episode 2 · DeepSeek V4.1 Flash Rediscovers CVEs for $2.4, Closing Gap with Frontier Models (2026-10-22, 2 posts)

Third-party benchmarks show DeepSeek V4.1 Flash rediscovers 87.5% of benchmark CVEs at $2.4 each—27x cheaper than Opus 5—leading commentators to say open models are rapidly closing the gap with frontier closed models in offensive cyber capability.

Episode 3 · DeepSeek V4.1 Flash 1017 jumps to 57% on Cyber Index, tops open-weight models (2026-10-24, 6 posts)

On Oct 24, Artificial Analysis updated its Cyber Index: DeepSeek's open-weight flagship V4.1 Flash, new version 1017, jumped from 41% to 57% (+16 points), becoming the top open-weight model and ranking second overall, behind only GPT-6 offered via a trusted access program. DeepSeek released the model the same day, self-reporting notable software engineering gains.

Confirmed

  • DeepSeek released DeepSeek-V4.1-Flash-1017 on Oct 24, just over five weeks after the initial version; officially described as a minor update that shortens task execution time and output length (per @dlweekly's summary of the announcement)
  • Official self-evaluation: 77% resolve rate on DeepSWE v1.1, surpassing Claude Opus 5
  • Scored 57% on the Cyber Index, up 16 points from 41%, topping available open models ahead of rivals like Grok (per Artificial Analysis)
  • Second overall, behind GPT-6 (available via trusted access, not openly available)
  • Cost of $0.42 per task: more than twice MiMo-V2.6-Pro (56%, $0.18) but roughly 4x cheaper than GPT-6 Sol (Daylight Blue variant)
  • Cyber Index methodology combines three real-repository evaluations of vulnerability discovery, reproduction and repair: CWE-Bench-AA (based on Collinear AI's CWE-bench), DeepsecBench-A, and more

Unconfirmed

  • The 77% DeepSWE result beating Claude Opus 5 is DeepSeek's own self-evaluation, not yet corroborated by third parties

Why it matters

  • An open-weight model now nears the strongest closed model on cyber defense tasks at roughly a quarter of the cost, offering strong value for security engineering automation
  • The index uses real codebase tasks rather than synthetic problems, making results a meaningful proxy for real-world vulnerability repair capability
  • A "minor update" delivering a 16-point jump plus software engineering gains reflects how steep the open-model capability curve is under rapid iteration