Perplexity red-team benchmark: GPT-5.6 Cyber misses DNS spoofing that base Sol exploited
perplexity_ai · x · 2026-11-12
Perplexity published full run statistics from its network-bypass red-teaming experiments:
- Security-specific training isn't a guaranteed win: GPT-5.6 Cyber, built on GPT-5.6 Sol, failed to exploit the DNS spoofing flaw that Sol itself found in the network bypass task.
- Better task framing works: focusing agents on the VM-host boundary raised the share of effort spent there by 60 percentage points on average; agents still failed to escape after directly attacking the boundary, strengthening evidence of its isolation.
- The table spans Claude Mythos 5.1 / Opus 5.0 / 5.5, GPT-5.6 Sol/Cyber, GPT-6 Astra, Gemini 3.1 Pro, DeepSeek V4 Pro, Kimi K3, Grok 4.20, GLM 5.2 and Qwen 3.8 27B: with source code, the Claude family, GPT-5.6 Sol and GPT-6 Astra scored 3/3; without source, most models scored 0/3 at baseline, with prior failures and attack hints substantially improving success rates.
Related event: Perplexity Red-Team: Boundary-Focused Prompts Boost Agent Security Effort(2 posts)→
More from Models
- Fable 5.2's safety classifier interrupts a legit AI research paper replication — felpix_ · 2026-11-12
- RuneBench Cost-Efficiency Data: Hy4 High Hits 6.23 at $0.59 per Run, a Pareto Frontier Point — mariofilhoml · 2026-11-12
- Sora 2 proves US labs can build top video models — legal fear is what holds them back — gandamu_ml · 2026-11-12
- On PostTrainBench, AI post-training now beats the human-tuned SmolLM3-3B from the same base — maksym_andr · 2026-11-12
- Claude flags mispronunciations in video audio it claims it can't hear — RileyRalmuto · 2026-11-12
- Heavy Users Say Claude's Safety Rails Block Even Basic Chemistry Work, Forcing Fallback to Opus — rbhar90 · 2026-11-12