Bengio team's 2017 paper resurfaces: sharp minima can generalize for deep nets
burny_tech · x · 2026-11-17
In a discussion involving Schmidhuber, burnytech resurfaced the 2017 Bengio-team paper "Sharp Minima Can Generalize For Deep Nets" (arXiv:1703.04933). It argues the popular flatness-generalization hypothesis is problematic for deep models: exploiting parameter-space symmetries of rectifier networks, one can construct equivalent models with arbitrarily sharper minima, and reparameterization can drastically change parameter geometry without affecting generalization. Most flatness notions thus can't directly explain deep network generalization.
Related event: Bengio Team's Old Paper Resurfaces: Sharp Minima Can Generalize(3 posts)→
More from Research
- Does the Hessian in LLM losses actually help? Debate cites Sharp Minima paper — burny_tech · 2026-11-17
- Sharp Minima Can Generalize for Deep Nets — and LLM Training Seems to Agree — burny_tech · 2026-11-17
- LLMs 101: free 2026 guide covers tokens to local deployment from first principles — CatAstro_Piyush · 2026-11-17
- NUS Computing names nine inaugural Ascent Fellows for early-career researchers — rishabh16_ · 2026-11-17
- Building a 100-question stereonet eval to test AI models' geology and 3D spatial reasoning — DynamicWebPaige · 2026-11-17
- RLAD Paper: Two-Player RL Teaches LLMs to Discover Reasoning Abstractions — burny_tech · 2026-11-17