scaling01: single AECI score useless, frontier labs should disclose full benchmarks
scaling01 · x · 2026-11-06
- scaling01 cites a "receipts" thread: Anthropic is on a steeper capability trend, and frontier labs should disclose internal capability levels—hiding them is itself a safety risk.
- He argues a single AECI score is useless now that models are superhuman at coding and math; labs must release individual benchmark scores and examples of emerging capabilities per domain.
- ECI rose from 160 to 170 this year with models generating Fields-medal-worthy proofs; he asks what 200 ECI models will do by end of next year.
Related event: scaling01: Single AECI Score Meaningless, Calls for Public Benchmarks(2 posts)→
More from AGI Musings
- Santa Fe Institute chronicles the rise of agent-based modeling in social science — sebkrier · 2026-11-06
- Correction: METR estimates Anthropic's capability growth accelerated 2x, driven by automated AI R&D — ronbodkin · 2026-11-06
- Researchers: at least half of new papers look like LLM-assembled copycats, and LLM reviewers love them — examachine · 2026-11-06
- AI 2027 modeler corrects math: RSI software R&D multiplier is 3.8x, not 4.5x — JeffLadish · 2026-11-06
- Why do people hate AI data centers? A Redditor's defense sparks debate — Subwooferrrs · 2026-11-06
- CV Pioneer Michael Black: Frontier Models Bring Classical Algorithms Back Into the Mix — Michael_J_Black · 2026-11-06