Microsoft researchers improve convergence with row norm and sampling tuning
Microsoft researchers integrate row norm into Dion's row selection, observing slight statistical improvements. They also find that scaling learning rate by 1/√f when sampling fraction f of rows yields about 1 centinat convergence gain.
2026-08-14 ~ 2026-08-14 · 2 related posts
- Scale learning rate by 1/sqrt(f) when sampling rows, Microsoft finds — JohnCLangford · 2026-08-14
- Microsoft folds Normuon row-norm into row selection for modest gains — JohnCLangford · 2026-08-14