Diffusion models emerge from associative memory: spurious states may signal generalization
DimaKrotov · x · 2026-11-02
A Nature Communications paper by Dmitry Krotov and colleagues interprets diffusion models through the lens of Dense Associative Memories (generalized Hopfield networks).
- Diffusion models are viewed as energy-based memories storing training data at local minima
- When training data exceeds critical storage capacity, new local minima — spurious states distinct from training samples — emerge
- As data grows, generation shifts from memorization to spurious states to generalization
- The authors suggest these "false memories" may actually signal emerging generative capability
More from Research
- Karpathy's nanochat: the best ChatGPT $100 can buy, near 60k GitHub stars — karpathy · 2026-11-02
- After reading 46 resources on agent evals, one author explains why her 95% score was misleading — blaizedsouza · 2026-11-02
- How to Loop Your Transformer: Meta/MIT arXiv paper on loop scaling laws — rohanpaul_ai · 2026-11-02
- Meta & MIT paper: looping transformer layers beats adding depth at fixed size — rohanpaul_ai · 2026-11-02
- AI tell-tale word 'delve' declining in latest NeurIPS papers, analysis shows — JungWooHa2 · 2026-11-02
- ByteDance Seed's Loop-μP paper draws 'Chinese loopmaxxers' buzz — scaling01 · 2026-11-02