Study: Optimal Optimizer Varies With Batch Size, Rankings Can Flip

Researchers from Princeton, Stanford and UCSD systematically scanned batch sizes and found that the best optimizer depends on batch size, with optimal scaling rules varying by setting and optimizer rankings flipping accordingly.

2026-10-10 ~ 2026-10-10 · 2 related posts

• Princeton-Stanford study: optimizer rankings flip with batch size; best scaling rule is setting-dependent — xingyudang · 2026-10-10
• Comprehensive sweep finds the best optimizer changes with batch size — aaron_defazio · 2026-10-10