RG view of generalization says neural nets learn scale-invariant correlation structure
burny_tech · x · 2026-07-29
The thread presents a renormalization-group (RG) view of why neural networks generalize.
Core claim
Neural nets generalize because they learn scale-invariant correlation structures in natural data, not because they memorize individual samples.
What the diagram argues
- Training and test examples can differ in microscopic details but still come from the same underlying system.
- Networks progressively discard irrelevant fine-grained details while preserving correlations that remain stable across scales.
- In RG terms, both training and unseen samples flow toward the same fixed point when they share the same correlation structure.
When generalization fails
Generalization breaks when unseen data comes from a different system, or when the relevant correlation structure changes across scales. In that case, samples map to a different representation and predictions fail.
Related event: Explaining Neural Network Generalization via Renormalization Group(2 posts)→
More from Research
- ICLR’s 2026 deadline joke lands with a NeurIPS timing twist — delliott · 2026-07-29
- Microsoft removes Mage Flow and points to a more efficient Qwen-VL-class encoder — Dante_77A · 2026-07-29
- Profluentbio is building protein foundation models for AI-designed therapeutics — nathanbenaich · 2026-07-29
- VisualPatchWorld: Code World Models for Efficient Planning — HKBU-KnowComp · 2026-07-29
- A staged diagnosis finds most short-text generation loss comes from the codec — ITMO · 2026-07-29
- Nature paper measures non-Gaussian order-parameter statistics across a phase transition — burny_tech · 2026-07-29