New research: standard SGD matches AdamW for LLM RL training, with far less memory overhead
zhaoran_wang · x · 2026-09-14
A new study challenges the default assumption that AdamW is required for training LLMs with reinforcement learning.
Key findings:
- During RLVR, vanilla SGD performs on par with AdamW while being far more memory-efficient, since AdamW must store both first and second gradient moments.
- The authors argue RL is a fundamentally different training regime from SFT: the variance of the second moment in RL is roughly 22x lower than in SFT, so most parameters share nearly the same effective step size, making adaptive learning rates largely unnecessary.
- Momentum also matters far less than assumed, suggesting RL fine-tuning can drop AdamW entirely for a sparser, cheaper optimizer setup.
Related event: Study: Standard SGD Can Replace AdamW in LLM RL Training(2 posts)→
More from Infra
- Intel plunges 29% in record one-day crash after guidance badly misses estimates — ivan_bezdomny · 2026-10-30
- Nebius runs auction-driven GPU pricing, lifting rates ~20% amid tight supply to 2028 — BenBajarin · 2026-10-30
- Bittensor's Terza Inference routes one API across miners' models on subnet 103 — const_reborn · 2026-10-30
- OpenRouter rebuilds provider routing: 46% fewer capacity errors, 16.9% faster TTFT — steipete · 2026-10-30
- NVIDIA's Nemotron 3.5 Super hits 51.78% on SWE-bench Pro with 4.6x fewer active params — dair_ai · 2026-10-30
- Bonsai Compresses Qwen 27B from 55.6GB to 5.9GB, Retains 98.2% Benchmark Performance — nicolechirps · 2026-10-30