New theory paper from Francis Bach's group analyzes WSD learning-rate schedules for least-squares problems
BachFrancis · x · 2026-10-21
Francis Bach shares the final paper from his postdoc, led by intern Noé at Inria: a theoretical analysis of WSD (Warmup-Stable-Decay) schedules for least-squares problems, deriving final SGD excess-risk bounds and comparing constant vs. linear schedules empirically over n=50 runs. Relevant to understanding why WSD works in LLM pretraining.
More from Research
- First Passage Attention: path extrema enable perfect long-range retrieval with 25% cache — Yuxuan Liu · 2026-10-21
- OpenVLHarness turns any MLLM into a visual reasoner, +9.9 points across 22 datasets — yining_hong · 2026-10-21
- Pedagogical RL from MIT and Laude Institute teaches models to generate learnable reasoning trajectories — lateinteraction · 2026-10-21
- Laude Institute's Slingshots Four ships 14 open-source projects on agent harnesses and open models — lateinteraction · 2026-10-21
- Stanford team launches ScholarCatalyst benchmark for inspiration retrieval — lateinteraction · 2026-10-21
- Three reasons AI is not conscious and may never be, per Nature essay — anilkseth · 2026-10-21