PSGD's QEQ update rule yields same rule as KL+AIRM
YouJiacheng · x · 2026-08-31
YouJiacheng notes that it is surprising PSGD's original objective with QEQ update gives exactly the same update rule as KL+AIRM. This finding adds to the interesting mathematical connections discussed in the previous post regarding the optimizer's cost function.
Related event: PSGD Optimizer Objective Found Identical to KL-Shampoo(3 posts)→
More from Research
- Efficient Coding Theory Predicts Synaptic Conductance, Explaining Bio Neural Energy Efficiency — jgvfwstone · 2026-08-31
- Yao Class Team Proposes Fast Weight Attention for Continual Learning — QuanquanGu · 2026-08-31
- Lean explained with TypeScript: Proving math via type checking — jedisct1 · 2026-08-31
- Why avoiding potholes is harder than avoiding pedestrians for AI — aakashgupta · 2026-08-31
- PSGD Optimization Algorithm Hailed as Superior and Ahead of the Curve — YouJiacheng · 2026-08-31
- PSGD cost function matches KL-shampoo objective exactly, author notes — YouJiacheng · 2026-08-31