Nature Communications paper links diffusion models to associative memory, framing spurious states as first sign of generative capability
DimaKrotov · x · 2026-11-02
Dmitry Krotov and colleagues published a new paper in Nature Communications revealing a deep connection between generative diffusion models and associative memory (DenseAM, an extension of Hopfield networks).
The paper identifies three phases of generated samples:
- Memorization: reproducing training examples verbatim
- Spurious states: discovering new patterns between stored memories while maintaining well-defined basins of attraction
- Generalization: generating genuinely novel samples with flat energy landscapes around them
Experiments varying the local-minimum/local-maximum difference (K) on the Memortization dataset show a clear phase transition from memorization to generalization as training set size grows. The counterintuitive twist: spurious states, traditionally seen as a flaw of associative memory, may actually be the first sign of generative capability — not a defect but a core enabler of broad generative architectures, with conclusions holding across architectures and datasets.
More from Research
- Mollick: expert-level work benchmarks are brutally hard, and ARC-AGI, GDPval, METR are saturated — emollick · 2026-11-02
- Rebuilding a Protein Domain Distribution Pipeline After a Backup Failure — AryHHAry · 2026-11-02
- NVIDIA's Elastic Latent Recurrence lets one 4B model trade loop count for accuracy: 18→29 on AA Index — burny_tech · 2026-11-02
- IBM's STAIR retriever cuts invalid references 65× and hits 82.6% accuracy by using document structure — solyarisoftware · 2026-11-02
- ReLift Hits SOTA Dense 3D Tracking on WorldTrack With 2.7x Faster Pipeline — Technion · 2026-11-02
- Why robotics "capability jumps" are just aligning tasks to pretrained LLM skills — robrombach · 2026-11-02