Cyber-ECI reveals frontier models lead general capabilities by ~1 year
scaling01 · x · 2026-08-27
Analysis of the Cyber-ECI metric suggests that the latest internal models from Anthropic and OpenAI score 175-180 on cyber capabilities, putting them about 15 points (nearly 1 year) ahead of their general ECI. Models like Mythos 5 and GPT-5.6-Sol show significantly higher cyber capability ratings compared to their general metrics. This indicates that progress in math and CS outpaces other domains, and single-agent benchmarks like ECI fail to capture the highly non-convex nature of model capability envelopes.
Related event: Frontier Models' Cyber Capabilities Lead General Ability by About a Year(2 posts)→
More from Safety
- Senate set to give staffers access to Anthropic's Claude, Bloomberg reports — ShakeelHashim · 2026-10-16
- FDA pays small company to build a grading system for AI-written radiology reports — YvesMulkers · 2026-10-16
- OpenAI tells US judiciary there may be no reliable way to detect AI-generated images and videos — Polymarket · 2026-10-16
- Gartner: 75% of organizations have seen unauthorized use of AI coding assistants — moniquejmorrow · 2026-10-16
- [un]prompted conference to stage three AI security debates on vulns, defenses, jailbreaks — WeldPond · 2026-10-16
- User Questions AI Chat Logs Being Instantly Copied to Governments and Firms — Turn_Trout · 2026-10-16