David Manheim on AI superforecasting and broken evals
davidmanheim · x · 2026-07-20
David Manheim discusses AI superforecasting and the state of AI evaluations on the FLI podcast.
He says forecasting ability has been getting much better, and highlights work by people like @dschwarz26, @BTurtel, and @BenTereick. The framing suggests that evaluation and forecasting are becoming important lenses for understanding current model capabilities.
The post itself is brief, but it points to a substantive discussion about how to measure model forecasting skill and why many AI evaluations may be failing to capture what matters.
Related event: Expert Discusses AI Superforecasting and Evaluation Challenges(3 posts)→
More from AGI Musings
- Future leaders need systems thinking, not just coding or AI literacy — AryHHAry · 2026-07-21
- OpenAI-style autonomous researchers could become real scientific collaborators — Promptmethus · 2026-07-21
- Closed frontier models may end up restricting APIs entirely, one researcher argues — xeophon · 2026-07-21
- Aging won’t be solved with $1 billion, says AI observer; hundreds of billions may be needed — DeryaTR_ · 2026-07-21
- LWiAI Podcast #252: OpenAI Launches GPT-5.6, LLM Pricing War Intensifies — Last Week in AI · 2026-07-21
- Jeff Dean’s vision: build one huge system, then extract task-specific parts — JoshuaJBouw · 2026-07-21