Comprehensive sweep finds the best optimizer changes with batch size
aaron_defazio · x · 2026-10-10
A comprehensive sweep by @xingyudang's team shows the answer to "which optimizer is best" depends on batch size: the optimal BSZ scaling rule varies with settings and cannot be derived from early theories like SDE approximations or convex bounds.
Related event: Study: Optimal Optimizer Varies With Batch Size, Rankings Can Flip(2 posts)→
More from Research
- OmniSeek turns Omni-LLMs into evidence-seeking agents, +15.5 points on VideoHolmes — mohitban47 · 2026-10-10
- Study: Parametric neural control differentiates top neural network models of primate visual cortex — Dr_Alex_Crimi · 2026-10-10
- Architecture papers reading list highlights Mamba-3 non-commutative state tracking advance — zmkzmkz · 2026-10-10
- Dex-One2Many: Real2Sim2Real Turns One Human Video Into Robots Generalizing Across Configurations — furongh · 2026-10-10
- DINOv2 and Qwen3 Embedding Spaces Aligned Without a Single Image-Caption Pair — NandoDF · 2026-10-10
- Coevolved robot communication transfers poorly to 3D: 1 success in 30 seeds — uv-mex · 2026-10-10