A Brief History of Learning in Imagination: How World Models Tackle RL's Sample Efficiency Problem
lukaszkaiser · x · 2026-09-04
Amplify Partners published a survey article, "A brief history of learning in imagination," endorsed by OpenAI co-founder Łukasz Kaiser as a great overview of RL with limited data.
Key points:
- Deep learning's wins rest on massive datasets (ImageNet made AlexNet possible), but real-world data is expensive in time and money — sample efficiency is a critical unlock for production ML.
- "Learning in imagination" is a model-based RL approach with two components: an action-conditioned world model that simulates the environment, and a policy that interacts with it.
- Its defining feature: the policy is trained solely on data generated by the world model, making environment data essentially free and cutting the need for costly real-world collection.
- The piece uses analogies (a volleyball player's perception-decision loop) and traces the history of the research line, spanning work by Eloi Alonso, Danijar Hafner, and related world-model efforts.
A solid primer on the world-models + RL route to sample efficiency.
More from AGI Musings
- Stanford CS cooling as EE/ME rise, a bet on semis and physical AI — appenz · 2026-09-04
- Mistake theory is a mistake: rationalists' charity keeps getting them rugpulled — wfithian · 2026-09-04
- Why Asking AI to Draw a Circle Can Cost More Effort Than Drawing It Yourself — pixlpa · 2026-09-04
- Why Are We Pushing Text as the AI Interface When Language Is a Terrible Way to Work? — pixlpa · 2026-09-04
- "Tax self-driving cars" pitched as the politically palatable congestion policy — NathanpmYoung · 2026-09-04
- The Future of Software, Part 2: When Software Recedes — manosaie · 2026-09-04