NAVER AI Lab's FOAM optimizer halves optimizer state while matching SOAP and Muon
naver-ai · hf · 2026-11-13
NAVER AI Lab introduced FOAM (Factored One-sided Adam-Moment), a lightweight SOAP variant that uses input-only preconditioning to drop the output-side Kronecker factor and skip one eigendecomposition, plus a factored Adam-moment estimator.
In LLM pre-training it matches strong second-order and Muon baselines while using half the optimizer state and running at Muon-class speed, answering whether SOAP's benefits can be kept at near-AdamW cost.
More from Infra
- MI50 32GB vs V100 32GB for local inference: MI50 costs ~1.5x less — Gold-Produce9591 · 2026-11-13
- Voice model tip: make sure your SIP trunk delivers 16kHz audio, not upsampled 8kHz — pbbakkum · 2026-11-13
- Open kernels are clearly written for Blackwell, hinting inference already runs on BW — teortaxesTex · 2026-11-13
- Nvidia suppliers grew 64.5% but Q4 guide is the real risk; five calls for Nov 17 — tengyanAI · 2026-11-13
- Railway pledges to cover ~$740K in AI tokens lost in October hack — shauntrennery · 2026-11-13
- Dukoi Turns Your Old Android Phone Into a Free App Server in 2 Minutes — Scobleizer · 2026-11-13