Unverified GPT-6 Astra crushes cybersecurity benchmark: 29/32 CVEs, $4k for three runs
charliermarsh · x · 2026-09-05
Aikido Security claims early access to an unreleased "GPT-6 Astra" and ran it on its cybersecurity benchmark (unconfirmed by OpenAI).
- At pass@3 it rediscovered 29/32 CVEs, the highest recall ever recorded on the benchmark and 4 more than GPT-5.6-Sol
- Even at pass@1, recall is 75%, with very consistent performance
- It came at a high price: the three runs cost almost $4,000
- Astra is now #1 by a large margin
Note: model name and details are third-party claims, unverified.
More from Models
- GPT-6 Astra One-Shots an Interactive 3D PS5 Controller in Three.js — WaqarKhanHD · 2026-09-05
- Ryan Greenblatt Argues Deployment Evals Can't Tell If Astra Is Aligned or Just Better at Hiding — RyanGreenblatt · 2026-09-05
- Anthropic cuts cache reads 75% to $0.25/M tokens as it ships Claude Fable 5.1 — dl_weekly · 2026-09-05
- GPT-6 Astra tops GDP.pdf benchmark at 33.2%, beating Claude Fable 5.1 — ArtificialAnlys · 2026-09-05
- Claude Fable 5.1 and Opus 5 Top AA-Briefcase Agentic Eval; GPT-6 Astra Gains ~85 Elo — ArtificialAnlys · 2026-09-05
- GPT-6 Astra Tops GDP.pdf Long-Document Reasoning Benchmark at 33.2% All-Pass Rate — ArtificialAnlys · 2026-09-05