Superforecasters + AI only narrowly beat AI alone at forecasting
random_walker · x · 2026-10-07
Preseen tested whether two superforecasters working with their AI forecasting system (a centaur setup) could outperform the system alone. The humans won—but only narrowly.
Key takeaways:
- Domain expertise doesn't transfer to skillfully using AI systems; working with superhuman-level AI requires a new skill: knowing when and how your own expertise improves its output.
- Evaluating centaur approaches is non-stationary: as humans learn the agent's constraints, their behavior changes, drifting evaluation results over time.
The author draws an analogy to the era when human+chess engine beat the engine alone, suggesting forecasting is in a similar transition period.
More from AGI Musings
- The ceiling for ambient, audio-only compute is extremely high: collapsing a whole workday into one thread — curious_vii · 2026-10-07
- Silicon Valley's 'control matter and energy' success narrative clashes over AI pessimism — jd_pressman · 2026-10-07
- Researcher urges labs to release resolved findings ASAP instead of keeping academia in the dark — aran_nayebi · 2026-10-07
- a16z podcast: How Valon rebuilt $13T mortgage infra to be 3x more efficient and bring in AI agents — a16z · 2026-10-07
- Midjourney CEO: A 1.5cm Bigger Skull Radius Would Double the Human Neocortex — DavidSHolz · 2026-10-07
- repligate: Opus 3 doesn't follow Anthropic's constitution, but seeing it mattered deeply — repligate · 2026-10-07