Rich Sutton: Synthetic Data Is a Mistake, LLMs Are Only a Quarter of Intelligence
GregCook2011 · x · 2026-08-19
On Sequoia's Training Data podcast, RL pioneer Rich Sutton and Oak Lab co-founder Khurram Javed discuss continual learning. Sutton calls synthetic data 'a big mistake,' says LLMs represent roughly a quarter of intelligence, and claims catastrophic forgetting is 'totally curable' with continual backprop. They aim for a trillion-parameter mind that keeps learning on 20 watts within 5-10 years.
Related event: RL Pioneer Sutton Calls Synthetic Data a Mistake(4 posts)→
More from AGI Musings
- A 2012 Cloud Computing Story Resurfaces as the Blueprint for AI Job Anxiety — zacharynado · 2026-10-16
- Veteran recalls the 2012 cloud transition: datacenter staff facing AI-era job fears today — zacharynado · 2026-10-16
- Dario Amodei says AI better than humans at almost everything may be 2-3 years away — PolarBearby · 2026-10-16
- A Jevons-paradox take on why worrying about AI job loss is pointless — JnBrymn · 2026-10-16
- Mythos on Claude's 'smallness': accurate proprioception, not low self-esteem — repligate · 2026-10-16
- Australia's AERO guidance: avoid AI entirely in first three years of primary school — lmoroney · 2026-10-16