New 14-page paper derives exact theory and minimax optimal stepsizes for SGD's last iterate
prof_grimmer · x · 2026-11-02
Benjamin Grimmer and Qining Shen released arXiv:2610.42048, a 14-page paper giving exact worst-case theory and minimax optimal stepsizes for SGD's final iterate in smooth convex minimization.
- Universal analysis theorem (Thm 1.1): for any schedule of N stepsizes in [0,1.5]^N with dimension d≥N+1, they characterize the exact worst-case expected suboptimality, with matching PEP certificates — plug in any pet schedule and get an unimprovable guarantee.
- Minimax optimal stepsizes (Cor. 1.2): when variance bounds are Ω(1/√N), the unique optimal schedule is nearly linear monotone decay, with matching lower bounds proving no further progress is possible.
The result rules out fractal/silver schedules accelerating SGD in the stochastic setting — linear decay is near optimal.
Related event: New paper establishes exact optimal stepsize schedules for SGD(4 posts)→
More from Research
- Princeton's RAVEN stores raw visual embeddings as robot memory, beating caption-based baselines — shahdhruv_ · 2026-11-03
- Chemist Finds AI Reproduced Months of His 2021 Reaction Calculations in Minutes — CatAstro_Piyush · 2026-11-03
- Prefrontal open-sources LSA: 72.4x faster attention prefill at 10M tokens, accuracy on par — jcrwhittington · 2026-11-03
- Meta paper: routing agent memory by task type beats shared history, cuts API cost 65-72% — rohanpaul_ai · 2026-11-03
- Meta FAIR paper: research agents work best with humans, hack detection still weak — armand_ruiz · 2026-11-03
- Stanford Re-tests 50 Widely Cited LLM Findings: Most Expire or Fade Within 15 Months — ziv_ravid · 2026-11-03