Networks aren't scale invariant — so why has scale invariance motivated so many useful methods?
jasondeanlee · x · 2026-11-10
Jianlin Chen notes L(w)=L(cw) doesn't hold in popular architectures (nets aren't scale invariant). Jason Dean Lee quotes and asks: given that, why have countless useful methods been motivated from scale invariance? A sharp question on the gap between theoretical motivation and actual architectures.
Related event: Networks Aren't Scale-Invariant, So Why Do Scale-Inspired Methods Work?(2 posts)→
More from Research
- Four Free Math Books for ML: Linear Algebra, Probability, Optimization, Information Theory — cneuralnetwork · 2026-11-10
- 4 free math books that actually matter for ML: linear algebra to information theory — cneuralnetwork · 2026-11-10
- Meta FAIR's e-means beats k-means in 1/10 epochs with 10-45x GPU speedups — burny_tech · 2026-11-10
- NeurIPS workshop paper: gradient-based data attribution can't reliably filter subliminal learning — ponguru · 2026-11-10
- Yang-Mills Working Group to Release OpenAI's Full Lean Code and Validation Tomorrow — burny_tech · 2026-11-10
- Interpretability researchers reminded: explanation matters, not just predictive power — burny_tech · 2026-11-10