Ramez Naam critiques AI Futures: elegant models mean little without real-world calibration
ramez · x · 2026-11-11
Ramez Naam followed up on his MTS podcast critique of AI Futures: he respects the team's forecasting discipline and admits the project has been excellent on many AI predictions, but argues its most important forecasts remain untested and cites specific issues like its use of METR time horizons. His core point: without empirical data to test models or calibrate parameters, you can build many models that look good on paper but don't match reality — so detailed real-world data must take precedence. David Manheim replied that key uncertainties are underdetermined by data, but absent per-domain verification, general predictive track records — where Daniel Kokotajlo beats almost everyone — should be the fallback.
Related event: Debate Erupts Over AI Timeline Forecasts as Data Meets Track Records(4 posts)→
More from AGI Musings
- Robin Li calls AGI a capital-fueled gamble, says Silicon Valley is betting billions on a myth — teortaxesTex · 2026-11-11
- Ex-OpenAI researcher bshlgrs: ~1/3 chance of superintelligence within a year, likely takeover — birchlse · 2026-11-11
- Apollo Research Head: Capabilities Faster, Safety Slower Than Hoped — and It's Intensifying — MariusHobbhahn · 2026-11-11
- "No revolutionary leap in 5 years": a skeptic's case that the AI industry is overhyped — imjustnewatai · 2026-11-11
- Europe's AI Test Is the Retrofit: Adding Cognition to Existing Industry Under Binding AI Act Deadlines — ingliguori · 2026-11-11
- Alignment progress lags far behind capabilities — nobody knows how to align superhuman models — MariusHobbhahn · 2026-11-11