Combining PSGD-Kron and KL-Shampoo yields an optimizer with no eig/inverse/solve
YouJiacheng · x · 2026-08-26
An interesting optimizer idea (work done by GPT Pro): combine PSGD-Kron's Σ^{-1/2} multiplicative update (Σ=E[ggᵀ]) with KL-Shampoo's KL objective, yielding an optimizer that needs no eigendecomposition, no matrix inversion, and no linear solves.
The method applies multiplicative updates to P with the objective KL(N(0, Σ), N(0, P⁻²)) — a promising route to cheaper second-order optimization.
Related event: GPT Pro Combines PSGD-Kron and KL-Shampoo into Decomposition-Free Optimizer(4 posts)→
More from Research
- NTU to host IAS Frontiers Conference on Geometry, Dynamics, and Learning (GDL'26) — FrnkNlsn · 2026-08-26
- DeepSeek V4 Review: Frontier at CVE Finding, Weak on Hallucination — teortaxesTex · 2026-08-26
- GaussVid: Sparse-View Gaussian Splatting enhanced with 3D-Aware Video Diffusion Priors — zhenjun_zhao · 2026-08-26
- LagrangeGS models dynamic 3DGS as a non-conservative Lagrangian system — zhenjun_zhao · 2026-08-26
- Misanthrope: A Privacy-Preserving Keypoint Detector that avoids detecting people — zhenjun_zhao · 2026-08-26
- KLTNet: Learning-Based Sparse Feature Tracking for Improved Monocular VIO — zhenjun_zhao · 2026-08-26