PSGD cost function matches KL-shampoo objective exactly, author notes
YouJiacheng · x · 2026-08-31
YouJiacheng points out that the online PCA cost function in section H.3 of the paper 'Curvature-Informed SGD via General Purpose Lie-Group Preconditioners' (2402.04553) is identical to the KL divergence objective in KL-shampoo. Interestingly, this cost was not interpreted as a KL divergence in the paper. The author asks if this cost is well-known in the online PCA literature.
Related event: PSGD Objective Found Mathematically Equivalent to KL-Shampoo(6 posts)→
More from Research
- Comprehension Audits: New Paper Proposes a Gate for AI-Written Code at Frontier Labs — ronbodkin · 2026-10-16
- Nature paper uses language models to map single neurons behind human language — raphaelmilliere · 2026-10-16
- Reviewer shares workflow: feeding the finished review — never the paper — to AI for a second pass — StingMeleoron · 2026-10-16
- 3SUM record falls to O(n^1.921), Lean-verified — but will it yield usable algorithms? — iskander · 2026-10-16
- Ablation studies to help Kaggle Gemma 4 competition participants — ayushthakur0 · 2026-10-16
- Experts put global AI catastrophe risk before 2100 at 5%, LEAP survey finds — ben_j_todd · 2026-10-16