Combining PSGD-Kron and KL-Shampoo yields an optimizer without eig/inverse/solve
YouJiacheng · x · 2026-08-26
The author combines PSGD-Kron's Σ^{-1/2} multiplicative update with KL-Shampoo's KL objective to propose a new optimizer that eliminates the need for eigen-decomposition, matrix inversion, or solving linear systems.
The core method involves multiplicative updates of P under the KL divergence objective KL(N(0, Σ), N(0, P^{-2})). The work was assisted by GPT Pro.
More from Research
- A 30-question deep dive into embeddings, vector search and retrieval — techNmak · 2026-08-26
- UBio-MolFM: Quantum-Accurate Simulation of Million-Atom Systems — 量子位 · 2026-08-26
- Deepak Pathak Demonstrates In-Context Learning for Robotics with One-Shot Video Prompts — deepakpathak · 2026-08-26
- OraRL: Efficient and Scalable RL for Video MLLMs — Yunheng Li · 2026-08-26
- Three hours of agentic research shows frontier LLMs can evade AI detectors — maier_ak · 2026-08-26
- How LLMs Self-Correct Mid-Generation: The Role of Reasoning RL and Instructions — dejanseo · 2026-08-26