GPT Pro synthesizes a new optimizer combining PSGD-Kron and KL-Shampoo
YouJiacheng · x · 2026-08-26
A researcher used GPT Pro to combine PSGD-Kron's Σ^{-1/2} multiplicative update (Σ=E[ggᵀ]) with KL-Shampoo's KL objective, yielding an optimizer that requires no eigendecomposition, matrix inversion, or linear solves.
- Core idea: multiplicative update of preconditioner P with objective KL(N(0, Σ), N(0, P⁻²)).
- The full report was synthesized by GPT, a concrete demonstration of LLM-driven research producing novel optimizer designs.
More from Research
- A 30-question deep dive into embeddings, vector search and retrieval — techNmak · 2026-08-26
- UBio-MolFM: Quantum-Accurate Simulation of Million-Atom Systems — 量子位 · 2026-08-26
- Deepak Pathak Demonstrates In-Context Learning for Robotics with One-Shot Video Prompts — deepakpathak · 2026-08-26
- OraRL: Efficient and Scalable RL for Video MLLMs — Yunheng Li · 2026-08-26
- Three hours of agentic research shows frontier LLMs can evade AI detectors — maier_ak · 2026-08-26
- How LLMs Self-Correct Mid-Generation: The Role of Reasoning RL and Instructions — dejanseo · 2026-08-26