Resurfaced Bengio paper: sharp minima can generalize for deep nets
burny_tech · x · 2026-11-17
In a discussion involving Schmidhuber, someone resurfaced the 2017 paper "Sharp Minima Can Generalize For Deep Nets" (arXiv:1703.04933) by Dinh, Razvan Pascanu, Samy Bengio, and Yoshua Bengio. It argues against the popular flat-minima generalization hypothesis: exploiting symmetries of ReLU networks' parameter space, one can build equivalent models at arbitrarily sharper minima, and reparameterization can drastically change parameter-space geometry without affecting generalization — so most notions of flatness cannot directly explain generalization in deep models.
More from Research
- Sharp minima debate continues: Hessian still seems to help for LLMs in practice — burny_tech · 2026-11-17
- Bengio team's 2017 paper resurfaces: sharp minima can generalize for deep nets — burny_tech · 2026-11-17
- LLMs 101: free 2026 guide covers tokens to local deployment from first principles — CatAstro_Piyush · 2026-11-17
- NUS Computing names nine inaugural Ascent Fellows for early-career researchers — rishabh16_ · 2026-11-17
- Building a 100-question stereonet eval to test AI models' geology and 3D spatial reasoning — DynamicWebPaige · 2026-11-17
- RLAD Paper: Two-Player RL Teaches LLMs to Discover Reasoning Abstractions — burny_tech · 2026-11-17