Ramez Naam: METR time-horizon and exponential models don't match OpenAI and GDM empirical data
clarejtbirch · x · 2026-11-12
Ramez Naam voices frustration that AI theory models — synthetic benchmarks like METR's time horizon and claims like 'ECI is exponential' — diverge dramatically from the best empirical data. Actual time horizons at OpenAI and GDM are much shorter, and OpenAI's don't appear to be rising exponentially. He concedes the real-world data is limited and argues for pressing for more granular, frequently updated evidence: if your models don't update on the evidence, why trust them?
Related event: Debate Erupts Over Credibility of AI Progress Forecasting Models(17 posts)→
More from AGI Musings
- Canada Tech Job Vacancies: 14 Roles Over 18 Quarters, Built on StatCan and Indeed Data — RichardsonDx · 2026-11-12
- 1.5-Year Study: Cultural Bias in AI Curiosity and Its Role in Multi-Agent Collaboration — radamihalcea · 2026-11-12
- Mollick: coding shows AI transforming a profession in under a year, far faster than historical adoption — emollick · 2026-11-12
- LessWrong: Why AGI Safety Strategy Needs Scenario-Based Strategic Foresight — JacquesThibs · 2026-11-12
- Physicist Slates Public Backlash Over Qantas, Canva and APS Job Losses — MJBiercuk · 2026-11-12
- Very few people can build agents for scientists, and that's the real bottleneck — nlarusstone · 2026-11-12