Orabona: the classic preconditioning story for full-matrix AdaGrad doesn't hold up

bremen79 · x · 2026-11-18

Optimization theorist Francesco Orabona published a new post challenging the deep learning mythology around preconditioning.

Directly relevant to the theoretical foundations of matrix-style optimizers like Shampoo.

Original post →

More from Research

Research channel →