Rich Sutton: Synthetic Data is a Mistake, The Goal is a Continually Learning Trillion-Parameter Mind
Training Data (Sequoia) · rss · 2026-08-18
RL pioneer Rich Sutton and his student Khurram Javed founded Oak Lab to build agents that learn continuously from their own experience. Sutton argues that continual learning is the norm and labels the current focus on synthetic data as "a big mistake." Their "Big World Hypothesis" suggests the world is too complex for frozen models, necessitating constant updates. Sutton views LLMs as an unanticipated breakthrough representing only a quarter of intelligence and believes catastrophic forgetting is curable via their continual backprop algorithm. Their goal is a trillion-parameter, 20-watt mind that learns continuously, a shift frontier labs struggle to make due to local minima.
More from AGI Musings
- Billions use search agents because search UX has long been broken — StefanoGogioso · 2026-08-18
- The real AI labor crisis: companies just stop replacing people who leave — VraserX · 2026-08-18
- MIT study: AI usage reports are biased, work use cases overestimated — nordicinst · 2026-08-18
- MIT study: Recursive self-improvement may be slower than expected — nordicinst · 2026-08-18
- VC view: AI as precursor to robotics, lacking consumer moment — arian_ghashghai · 2026-08-18
- Oxford Professor: Leaders Should Ignore AI Predictions and Shape the Future — AnnaCiaunica · 2026-08-18