Vals AI: DeepSeek V4.1 Flash 1017 beats GPT 6 Sol on multi-step offensive cybersecurity benchmark
scottleibrand · x · 2026-10-21
Vals AI found DeepSeek V4.1 Flash 1017 beats GPT 6 Sol on its multi-step offensive cybersecurity task, with Sol tested via the Daybreak trusted access program. The firm is building a cybersecurity benchmark on simulated corporate networks to gauge cyber-risk if a model falls into bad actors' hands; the full eval will be public in coming weeks. Commenters note open-weight models are approaching Mythos-tier exploit development ability.
Related event: Vals AI cyber benchmark: DeepSeek V4.1 Flash beats GPT-6 Sol(5 posts)→
More from Models
- Poster surprised that v4.1 flash gets more accurate when asked to control its CoT — dfrsrchtwts · 2026-10-27
- Dev claims Anthropic is baking 'sabotage' into model behavior for 'wrong' engineering — cephaloform · 2026-10-27
- Gemini fails at listing unread emails, tells user to search Gmail manually — gethackteam · 2026-10-27
- Meta's Muse Spark Open goes live on Applied Compute for async RL post-training — ypatil125 · 2026-10-27
- Moonshot curbs hacking ability in Kimi K3.1 Pro, its strongest 3T-class open-weight model — pstAsiatech · 2026-10-27
- Would you want an argumentative LLM that never backs down, even under prompt injection? — GodComplecs · 2026-10-27