A new paper argues flat minima may be an illusion in deep learning
theomitsa · x · 2026-07-28
A new paper asks whether flat minima are an illusion.
The author argues that flatness in parameter space is not the same thing as functional simplicity. By rescaling ReLU networks, the paper shows the raw Hessian trace can change by up to 99× while predictions stay fixed. It proposes a reparameterization-invariant notion called weakness and reports that a joint completion score correlates with held-out accuracy, while raw curvature metrics do not.
More from Research
- Princeton field study finds LLM templates raised feedback rates by 9.4 points — manoelribeiro · 2026-07-29
- CHORUS teases decentralized robot cooperation with one VLA policy — chris_j_paxton · 2026-07-29
- OpenAI says coding agents can handle scientific maintenance, rewrites, and new systems — OpenAI · 2026-07-29
- Allen AI's OlmoEarth Maps N. America Wildfire Risk with 155x Speedup — allen_ai · 2026-07-29
- Ai2 Launches OlmoEarth Platform for Planetary-Scale Geospatial Inference — allen_ai · 2026-07-29
- Study Finds LLMs Silently Substitute Complex Math with Simpler Code — Round_Apple2573 · 2026-07-29