Resurfaced Bengio paper: sharp minima can generalize for deep nets

burny_tech · x · 2026-11-17

In a discussion involving Schmidhuber, someone resurfaced the 2017 paper "Sharp Minima Can Generalize For Deep Nets" (arXiv:1703.04933) by Dinh, Razvan Pascanu, Samy Bengio, and Yoshua Bengio. It argues against the popular flat-minima generalization hypothesis: exploiting symmetries of ReLU networks' parameter space, one can build equivalent models at arbitrarily sharper minima, and reparameterization can drastically change parameter-space geometry without affecting generalization — so most notions of flatness cannot directly explain generalization in deep models.

Original post →

More from Research

Research channel →