80,000 Hours: the intelligence explosion can't happen inside a data centre
rickasaurus · x · 2026-09-15
An 80,000 Hours piece by Tom Reed argues full AI R&D automation won't rapidly yield domain-general superintelligence:
- You can't get good at most things without practice, and AI labs lack the data models need to practice most domains.
- This can't be fixed with sample efficiency — in most cases the relevant data doesn't exist at all — nor with simulations or synthetic data.
- Superintelligence in most non-coding domains therefore depends on deploying AI throughout the real economy to generate the needed signal.
- An isolated data centre of automated geniuses would produce only a "Goodhart Singularity": metrics optimized, real capability stalled.
The resharing author adds that RL likely won't get us there, with beyond-size-scaling gains confined to areas labs focus on intensely.
Related event: Doubts grow over whether RL can lead to AGI(2 posts)→
More from AGI Musings
- Solo dev argues LLMs shouldn't be the default center of an agent runtime — HmmmThisIsOdd · 2026-09-15
- "Models are slowing down" narrative disputed, as Sabine Hossenfelder argues labs slow AI to dodge lawsuits — zetalyrae · 2026-09-15
- Oxford's Carissa Véliz: Better Science, Not AI Slop Tainting Academic Literature — CarissaVeliz · 2026-09-15
- Jack Clark: stories about AI literally become training data for the next models — jackclarkSF · 2026-09-15
- Jensen Huang revisits the AI predictions that never came true at All-In Summit 2026 — tetsuoai · 2026-09-15
- Critics say "Responsible Scaling Policies" is now "Pacing the Frontier" — same old evals — DavidSKrueger · 2026-09-15