Vals AI: DeepSeek V4.1 Flash beats GPT 6 Sol on offensive cyber benchmark, open weights favor defenders
sayashk · x · 2026-10-21
Security researcher Joshua Saxe argues open-weights models good at dual-use cyber tasks are net positive for cyber-defense, citing a Vals AI finding.
- Vals AI's multi-step offensive cybersecurity benchmark on simulated corporate networks found DeepSeek V4.1 Flash 1017 beats GPT 6 Sol (tested via the Daybreak trusted access program); full eval coming in weeks.
- Attacker productivity gains from AI (2022–26) appear small, near statistical noise, while defenders show measurable progress using models to find and fix bugs in high-leverage codebases.
- Defenders struggle to access the best unguardrailed closed-weight cyber models, and their tokens are far pricier than open-weights equivalents — prohibitive for cost-center CISO orgs.
- Open weights will ultimately be needed on-prem, on air-gapped networks and small devices to detect breaches and attacker implants.
More from Models
- Ternary-Bonsai-2-9B: 2-bit ternary vision-language model trends on Hugging Face for on-device use — prism-ml · 2026-10-21
- Grok Bot DAU hits new peak as users report it books flights and pays bills — elonmusk · 2026-10-21
- Interconnects: China proposes export pre-approval for frontier models as DHS bars DeepSeek from federal procurement — natolambert · 2026-10-21
- Researcher urges the community to stop using PickScore as a reward model, recommends HPSv3 instead — gowthami_s · 2026-10-21
- OpenAI boosts COBOL, Fortran and Visual Basic support to modernize legacy enterprise systems — CurieuxExplorer · 2026-10-21
- NVIDIA Hosts Ask-the-Experts Session on Nemotron 3.5 Super VL — NVIDIAAI · 2026-10-21