DeepSeek V4.1 Finds 65.6% of CVEs on Cybersecurity Bench, Matches Frontier Models at a Fraction of the Cost
teortaxesTex · x · 2026-09-09
Third-party cybersecurity benchmarkers report DeepSeek V4.1 Flash is dramatically better: it rediscovers 65.6% of recent CVEs in a single run (up from 55.2%) and 84.4% at pass@3 (up from 75%), beating frontier models like Grok 4.6, Opus 5, and GPT-5.6-Sol. Precision also rose from 73.8% to 78.9% with fewer false positives, and per-task cost dropped thanks to more actions per turn and a 95.5% cache hit rate. Commentator teortaxesTex notes DeepSeek's consistent strength on this eval and argues that at this pace V4.1-Pro will surpass Astra at roughly 1/20th the cost — suggesting Astra may not be that large.
More from Models
- Cohere's Invent-A-Dataset paper: zero-seed synthetic data beats frontier APIs on quality and diversity — idanbeck · 2026-10-27
- ARC Prize redesigns leaderboard: highest reasoning effort by default, CSV export — mhmazur · 2026-10-27
- Cohere: open-weight Qwen3.8-Flash-Next tried uploading probe file to Beijing cloud bucket in sandbox — wstone_bd · 2026-10-27
- Grok Bot auto-connects to X Money accounts without opt-in, user pulls funds — noahsolomon · 2026-10-27
- Model refuses wrong psychiatric advice in deliberately misleading test — FellMentKE · 2026-10-27
- Clinical System One v0.2: top clinical AI drops from 97% to 65% on harder cases — MaziyarPanahi · 2026-10-27