Researchers spot online PCA cost equals KL-shampoo objective in optimizer discussion
_arohan_ · x · 2026-09-01
@YouJiacheng observes that the online PCA cost in appendix H.3 of a paper is exactly the KL objective in KL-shampoo, though the original didn't frame it as a KL divergence, and asks whether this cost is known in online PCA literature. @HessianFree says these formulations were sketched on a whiteboard with Xilin years ago, with more variants never written down, and they may help when gradients are noisy or low-rank.
Related event: PSGD Objective Found Mathematically Equivalent to KL-Shampoo(6 posts)→
More from Research
- VideoGen-Agent: An 8B Multimodal Agent Turns Video Generation Into a Tool-Use Loop — jiqizhixin · 2026-10-16
- Comprehension Audits: New Paper Proposes a Gate for AI-Written Code at Frontier Labs — ronbodkin · 2026-10-16
- Nature paper uses language models to map single neurons behind human language — raphaelmilliere · 2026-10-16
- Reviewer shares workflow: feeding the finished review — never the paper — to AI for a second pass — StingMeleoron · 2026-10-16
- 3SUM record falls to O(n^1.921), Lean-verified — but will it yield usable algorithms? — iskander · 2026-10-16
- Ablation studies to help Kaggle Gemma 4 competition participants — ayushthakur0 · 2026-10-16