$25k AI Forecasting Contest: How to Make Top Models Disagree
tshevl · x · 2026-08-22
A $25,000 contest was held to discover where the best AI forecasting systems disagree. It attracted superforecasters and hedge fund quants probing for "wedge questions".
Motivation:
- Many questions are "solved" (e.g., nuclear explosion probability), leading to similar predictions.
- Some questions are intractable due to aleatoric uncertainty (e.g., stock prices), where models just agree on being uncertain.
Winning Tactics:
- Exploit information asymmetries: Forecasting Hugging Face downloads favored models with API access.
- Identify discontinuities: Predicting outcomes during rule changes (e.g., NIST classifications).
- Test decomposition: Asking about complex events (e.g., beating a leaderboard) forces problem breakdown.
More from AGI Musings
- AI Interaction Will Leap from Text to Mixed Reality — mrjonfinger · 2026-08-22
- DarkHaven alliance forming to fight AI doomers — beffjezos · 2026-08-22
- The irony of AI progress: computers may be the first to prove P=NP — EricBuess · 2026-08-22
- Philosopher Eric Schwitzgebel to Discuss Robot Consciousness at WorldCon — eschwitz · 2026-08-22
- "Nobody has priced in the Singularity": a viral satirical thread imagines post-AGI daily life — louisvarge · 2026-08-22
- Survey Shows Young People Deeply Distrust AI CEOs — GaryMarcus · 2026-08-22