Bengio et al. classic: sharp minima can generalize, flatness hypothesis flawed
burny_tech · x · 2026-11-10
- X user burnytech shares the 2017 paper Sharp Minima Can Generalize For Deep Nets (arXiv:1703.04933) by Laurent Dinh, Razvan Pascanu, Samy Bengio, and Yoshua Bengio
- The popular hypothesis holds that flat minima found by SGD explain good generalization
- The paper argues most notions of flatness are problematic for deep models and cannot directly explain generalization
- Key argument: exploiting the parameter-space symmetries of rectifier networks, one can construct equivalent models with arbitrarily sharper minima; reparametrization can drastically change parameter geometry without affecting generalization
Related event: Classic Paper by Dinh et al. Shows Sharp Minima Can Generalize(2 posts)→
More from Research
- PSI claims proof of Aaronson-Ambainis conjecture using a research system built on Claude — burny_tech · 2026-11-10
- PNAS Nexus paper demonstrates experience-based training in a living tissue model — burny_tech · 2026-11-10
- NVIDIA paper: masking ~10 'intrusion heads' recovers 57% of distractor accuracy loss — rohanpaul_ai · 2026-11-10
- The Five Differential Operators of Physics, All Built From One Symbol: Nabla — burny_tech · 2026-11-10
- Control Limit Hypothesis: High Dopamine in Schizophrenia Is Compensation, Not Cause — burny_tech · 2026-11-10
- Interp Researcher's 2026 Advice: Be Ambitious, Pragmatic, and Go Meta — burny_tech · 2026-11-10