2020 physics paper links learning rate, Hessian and gradient noise via fluctuation-dissipation theorem
burny_tech · x · 2026-10-28
Liu Ziyin resurfaces his 2020 ICML paper (arXiv:2012.03636) arguing recent edge-of-stability research follows from theoretical physics. Using the fluctuation-dissipation theorem, it derives a direct connection between learning rates, the Hessian and gradient noise under stochastic training, giving the stationary distribution of discrete-time SGD with and without momentum — showing discrete-time noise is distorted and far larger than continuous-time theory predicts, with applications to minibatch noise, escape from sharp minima, and second-order methods including Adam.
More from Research
- UltraVBERT: a 247M-parameter vision-language encoder for visual document retrieval — kalyan_kpl · 2026-10-28
- Anders Sandberg on a wild 48 hours: group classification formalized, Kakeya, RCE, AI incident — anderssandberg · 2026-10-28
- EvidenceConnect boosts multimodal perception without zooming, sweeping 8 benchmarks — inclusionAI · 2026-10-28
- What 200 years of philology reveals about LLM confabulation's structural root — vasilisvj · 2026-10-28
- Map-based localization hits 1.9cm RMSE across EuRoC flights in visloc-rs — rsasaki0109 · 2026-10-28
- BeeNara v0.4: 332MB local doc sorter hits 91.5% on unseen folders — razer_psycho · 2026-10-28