Optimizer Sweep Shows Best Choice Varies With Batch Size; Shampoo Variants Lead
zacharynado · x · 2026-10-12
- A comprehensive optimizer sweep by xingyudang et al. finds the "best optimizer" answer changes with batch size, and the optimal BSZ scaling rule depends on the setting — it can't be derived from early theories like SDE or convex bounds.
- arohan quips that the top 3 are all Shampoo and its variants, directing readers to the original author @GuptaVineetG.
Related event: Study Finds Best Optimizer Changes with Batch Size(3 posts)→
More from Research
- Landmark study of 12M single cells links T cell ratio to immune aging and mortality — EricTopol · 2026-10-12
- SocioVerse2 from Fudan and Oxford brings counterfactual intervention to LLM social simulation — jiqizhixin · 2026-10-12
- 12M single cells, 2,600 people: seminal immune aging paper yields practical blood test — EricTopol · 2026-10-12
- Community + AI agents push integer multiplication κ bound up 69% in a day via E8 geometry — CatAstro_Piyush · 2026-10-12
- New arXiv paper: density ratio estimation via Stein displacement fields — _onionesque · 2026-10-12
- JaxAHT v1.1 released: JAX-based Ad Hoc Teamwork benchmark with Hanabi and LBF tasks — PeterStone_TX · 2026-10-12