Researchers spot online PCA cost equals KL-shampoo objective in optimizer discussion
_arohan_ · x · 2026-09-01
@YouJiacheng observes that the online PCA cost in appendix H.3 of a paper is exactly the KL objective in KL-shampoo, though the original didn't frame it as a KL divergence, and asks whether this cost is known in online PCA literature. @HessianFree says these formulations were sketched on a whiteboard with Xilin years ago, with more variants never written down, and they may help when gradients are noisy or low-rank.
More from Research
- Daphne Koller: AI won't cure disease soon, human biology understanding is the bottleneck — pmddomingos · 2026-09-01
- Asymmetric Actor-Critic criticized for blocking gradients and causing robot gait issues — yacineMTB · 2026-09-01
- Gait Training Tip: Use Rolling Avg Velocity Reward — yacineMTB · 2026-09-01
- TMLR Emphasizes Clear Writing in New Acceptance Criteria — Aaroth · 2026-09-01
- Chollet debunks "100% on ARC-AGI-3" claim: only on easy public set — fchollet · 2026-09-01
- Author Clarifies AI Detection Study Tests Pangram — TuhinChakr · 2026-09-01