OpenAI's 96 ECI benchmarks skew to math and AI research; hard multimodal and cybersecurity tests are rare

scaling01 · x · 2026-11-19

OpenAI data shows model time horizons doubling every 3.8 months, and the 96 benchmarks behind its ECI reveal its priorities: math and AI research first, then science, professional work and engineering.

Benchmark counts by category (GPT-6.2 Astra at 170.3 as reference): Science 18, software/hardware engineering 16, professional work 10, language & reasoning 9, multimodal 9, math 8, AI research 8, computer/tool use 7, games 7, cybersecurity only 4. The poster is surprised how few hard multimodal, game and cybersecurity benchmarks exist.

Original post →

More from Models

Models channel →