Rich Sutton: Synthetic Data Is a Mistake, LLMs Are Only a Quarter of Intelligence
GregCook2011 · x · 2026-08-19
On Sequoia's Training Data podcast, RL pioneer Rich Sutton and Oak Lab co-founder Khurram Javed discuss continual learning. Sutton calls synthetic data 'a big mistake,' says LLMs represent roughly a quarter of intelligence, and claims catastrophic forgetting is 'totally curable' with continual backprop. They aim for a trillion-parameter mind that keeps learning on 20 watts within 5-10 years.
More from AGI Musings
- View: Intelligent representation choice yields high sample efficiency — MarvinTBaumann · 2026-08-19
- AI integration into daily life poses personal risks beyond economics — sarahookr · 2026-08-19
- No data centers in my backyard: Wisconsin residents push back on the AI buildout — ivan_bezdomny · 2026-08-19
- AI billing stuck in Edison's era: Shift from token counts to value-based pricing — AccBalanced · 2026-08-19
- Jeff Clune talks open-endedness and recursive self-improving AI at ALIFE 2026 — BertChakovsky · 2026-08-19
- Recursive Self-Improvement vs. Reward Hacking: 成功定义一切 — beenwrekt · 2026-08-19