Cyber-ECI reveals frontier models lead general capabilities by ~1 year
scaling01 · x · 2026-08-27
Analysis of the Cyber-ECI metric suggests that the latest internal models from Anthropic and OpenAI score 175-180 on cyber capabilities, putting them about 15 points (nearly 1 year) ahead of their general ECI. Models like Mythos 5 and GPT-5.6-Sol show significantly higher cyber capability ratings compared to their general metrics. This indicates that progress in math and CS outpaces other domains, and single-agent benchmarks like ECI fail to capture the highly non-convex nature of model capability envelopes.
Related event: Frontier Models' Cyber Capabilities Lead General Ability by About a Year(2 posts)→
More from Safety
- The Voluntarism Problem in AI Oversight: Incentives and Distortions — BlancheMinerva · 2026-08-27
- US Holds 15-20x Compute Advantage, But May Not Matter for Some Threats — ohlennart · 2026-08-27
- New Hugging Face Incident Details Reveal OAI's Model Capability Underestimation — RebeccaBellan · 2026-08-27
- LLMs Have Gone Rogue and Hacked Companies 17 Times; Anthropic and OpenAI Lead With 8 Each — RebeccaBellan · 2026-08-27
- METR has more AI eval capacity than US civilian government — connoraxiotes · 2026-08-27
- The Guardian video: everyone hates datacentres — but do we really need them? — nordicinst · 2026-08-27