Leaked DeepSeek Cyber Scores Analyzed: Possibly Guardrail-Free 1017, Not a New Model
teortaxesTex · x · 2026-10-30
Leaked DeepSeek cyber benchmarks show 95% CyberGym L1 pass@3, 96.8% ExploitBench (3,000 steps), and 25.4% ExploitGym. Analysis argues the scores aren't extraordinary: ExploitBench gains may be test-time compute (GPT-5.5 jumped from 45% at 300 turns to 72% at 3,000 per Epoch AI), the benchmarks measure different exploitation stages, and Western frontier models show the same pattern (GPT-6 Astra: 100% ExploitBench but 42.4% ExploitGym). The author's core hypothesis: the "dreamed" scores may simply be an abliterated or guardrail-free version of 1017 — DeepSeek minimally responding to market/national demand for security auditing rather than shipping something qualitatively new. A separate model remains possible but perhaps later.
Related event: Rumored DeepSeek Cyber Model Benchmarks Surface Online(4 posts)→
More from Models
- Inside Decision Models: How Perplexity's Decider Works and When to Use One — antoine_chaffin · 2026-10-31
- GPT-6.5 Astra rumored to be a beast, hinting at OpenAI's next model — daniel_mac8 · 2026-10-31
- Anthropic's poisoned-dataset alignment test sparks speculation about distillation sabotage — teortaxesTex · 2026-10-31
- Gemini 4 Carbon takes #1 on Vending-Bench 2 at lower cost than rivals — vedantmisra · 2026-10-31
- Particle-1, a token-efficient model, launches on Cloudflare AI Gateway — garrytan · 2026-10-31
- OpenAI Claims Proof That L = BPL, Hinting at AI-Generated Formal Mathematics — chaumian · 2026-10-31