Diffusion models as associative memory: Nature Communications paper maps 3 generation phases
DimaKrotov · x · 2026-11-02
A new Nature Communications paper by Dima Krotov and collaborators reveals a deep connection between generative diffusion models and Associative Memory, moving toward a 'physical' theory of generative models.
The work identifies 3 phases of generated samples:
- Memorization: reproducing training examples verbatim
- Spurious states: discovering new patterns between stored memories while maintaining a well-defined basin of attraction around those samples
- Generalization: generating genuinely novel samples with flat energy landscapes around them
The surprising twist: spurious states, traditionally considered a flaw of Associative Memory, may actually be the first sign of generative capabilities.
More from Research
- Karpathy's nanochat: the best ChatGPT $100 can buy, near 60k GitHub stars — karpathy · 2026-11-02
- After reading 46 resources on agent evals, one author explains why her 95% score was misleading — blaizedsouza · 2026-11-02
- How to Loop Your Transformer: Meta/MIT arXiv paper on loop scaling laws — rohanpaul_ai · 2026-11-02
- Meta & MIT paper: looping transformer layers beats adding depth at fixed size — rohanpaul_ai · 2026-11-02
- AI tell-tale word 'delve' declining in latest NeurIPS papers, analysis shows — JungWooHa2 · 2026-11-02
- ByteDance Seed's Loop-μP paper draws 'Chinese loopmaxxers' buzz — scaling01 · 2026-11-02