PSGD cost function matches KL-shampoo objective exactly, author notes
YouJiacheng · x · 2026-08-31
YouJiacheng points out that the online PCA cost function in section H.3 of the paper 'Curvature-Informed SGD via General Purpose Lie-Group Preconditioners' (2402.04553) is identical to the KL divergence objective in KL-shampoo. Interestingly, this cost was not interpreted as a KL divergence in the paper. The author asks if this cost is well-known in the online PCA literature.
Related event: PSGD Optimizer Objective Found Identical to KL-Shampoo(3 posts)→
More from Research
- Efficient Coding Theory Predicts Synaptic Conductance, Explaining Bio Neural Energy Efficiency — jgvfwstone · 2026-08-31
- Yao Class Team Proposes Fast Weight Attention for Continual Learning — QuanquanGu · 2026-08-31
- Lean explained with TypeScript: Proving math via type checking — jedisct1 · 2026-08-31
- Why avoiding potholes is harder than avoiding pedestrians for AI — aakashgupta · 2026-08-31
- PSGD Optimization Algorithm Hailed as Superior and Ahead of the Curve — YouJiacheng · 2026-08-31
- VLANeXt codebase release reveals recipes for building strong VLA models — ccloy · 2026-08-31