Bengio team's 2017 paper resurfaces: sharp minima can generalize for deep nets

burny_tech · x · 2026-11-17

In a discussion involving Schmidhuber, burnytech resurfaced the 2017 Bengio-team paper "Sharp Minima Can Generalize For Deep Nets" (arXiv:1703.04933). It argues the popular flatness-generalization hypothesis is problematic for deep models: exploiting parameter-space symmetries of rectifier networks, one can construct equivalent models with arbitrarily sharper minima, and reparameterization can drastically change parameter geometry without affecting generalization. Most flatness notions thus can't directly explain deep network generalization.

Related event: Bengio Team's Old Paper Resurfaces: Sharp Minima Can Generalize(3 posts)→

Original post →

More from Research

Research channel →