OpenAI models sweep Vercel's stealth DeepsecBench cyber benchmark, GPT-6.2 Astra tops at 61.22
LauraModiano · x · 2026-11-18
rauchj highlights that OpenAI has shipped what he calls the most cybersecurity-capable models in the world.
The key evidence is Vercel's newly surfaced DeepsecBench, a "stealth" cyber benchmark built on the deepsec cyber harness that measures how well models find security vulnerabilities in application code, combining recall and precision into a single score. Crucially, no lab has had access to the underlying codebases to train on, ruling out data contamination.
Leaderboard highlights:
- GPT-6.2 Astra (xhigh reasoning) tops the board at 61.22, with 56.2% recall and 95.0% precision, costing $59.85 in 6m26s
- The top 6 spots are all taken by different OpenAI GPT-6-family configurations; GPT-6.2 Astra at medium effort scores 53.81
- Claude Sonnet 5.5 is the best non-OpenAI entry at 37.50
Related event: OpenAI Releases GPT-6.2 Astra with Major Cybersecurity Gains(4 posts)→
More from Models
- Raschka: benchmarking new LLMs is like using IQ tests for hiring — rasbt · 2026-11-18
- Open-source Karpo-Decision-31B tops 300-model Decision Index with zero-token, one-forward-pass answers — rohanpaul_ai · 2026-11-18
- GPT-6.2 backlash is real user experience, not astroturf: dev says Astra has quirks but Sol is a solid daily driver — pvncher · 2026-11-18
- LLM protein-folding test on fresh PDB entry lands near-random structures — MoAlQuraishi · 2026-11-18
- Per-task costs span 20x: GPT-6.2 Sol at $2.63 vs Claude Fable 5.2 at $54.25 — haider1 · 2026-11-18
- Every's founder pushes back: we re-tested the model to the wire, OpenAI gave us no heads-up — kieranklaassen · 2026-11-18