Sander Dieleman on Diffusion Models, Typicality, and Why ML Breakthroughs Are Closer Than They Look
sedielem · x · 2026-09-08
Google DeepMind's Sander Dieleman — known for work on AlphaGo, WaveNet, and the diffusion models behind today's image and video generators — joins Fundamental CSO Marta Garnelo on the First Principles podcast.
Key points:
- Evolution of diffusion models: how diffusion became the backbone of modern generative systems, and what remains poorly understood about it.
- Hidden connections between paradigms: many ML breakthroughs that look distant in time are far closer in substance than they appear.
- Intuition breaks in high dimensions: generative AI is fundamentally about probability, abstraction, and compression — areas where human intuition systematically fails.
- Writing as a research tool: his widely-read blog isn't just output; writing and teaching function as instruments of scientific discovery.
The conversation also touches on music generation, typicality, his Kaggle past, and lessons from restricted Boltzmann machines.
More from Models
- Jensen Huang confirms GPT-6 Astra trained on 100K+ Grace Blackwell NVL72 systems — rohanpaul_ai · 2026-09-08
- 1,665 model runs across 27 repos: open-source models beat closed ones at security audit, dev claims — Fluffy-Ad-889 · 2026-09-08
- Blogger claims Google's Astra has 'basically solved' computer use — infoxiao · 2026-09-08
- Looped Transformer hype: Nanbeige4.2-3B beats 12B models on agent benchmarks — alexcovo_eth · 2026-09-08
- User Calls Claude's Refusals 'Anxiety Attacks,' Welcomes Astra as Alternative — ctjlewis · 2026-09-08
- Qwen 27B q8 vs bf16 on a DGX Spark: is the 1% token difference worth the memory? — superSmitty9999 · 2026-09-08