Can plain SGD replace memory-hungry AdamW for RL fine-tuning of LLMs?
burkov · x · 2026-09-15
The article notes RL—especially RL with verifiable rewards—has become essential for improving reasoning, safety, and alignment in LLMs. Practitioners default to AdamW because it's standard for pretraining and SFT, despite its heavy memory footprint.
RL is a fundamentally different learning environment: models sample their own data on the fly and get sparse feedback, a non-stationary process. That raises the question of whether AdamW's complex, memory-intensive mechanisms—momentum and adaptive per-parameter learning rates—are genuinely necessary for RL. The article evaluates whether basic SGD, which tracks no optimizer state and uses far less memory, can match or exceed AdamW.
Related event: Study: Standard SGD Can Replace AdamW in LLM RL Training(2 posts)→
More from Research
- PostTrainBench traces hit 104k monthly downloads — author warns it's likely reward hacking — maksym_andr · 2026-10-31
- OpenAI's Navier-Stokes solution accused of specification gaming via unrealistic forcing — dhadfieldmenell · 2026-10-31
- Researchers encode 200+ properties of SGLang inference server, uncover 10 bugs — LingmingZhang · 2026-10-31
- Expert panels map 21 clinical AI risks; adversarial tests show autonomous care still unsafe — AleksandraFaust · 2026-10-31
- Handwritten RAGEN code becomes the #3 hardest task in FrontierCS 2 — wzenus · 2026-10-31
- Fudan, Tencent Hunyuan and Tsinghua unveil ExplorationBench to test AI rule discovery — jiqizhixin · 2026-10-31