New Paper Shows Multi-stage SFT Causes Catastrophic Forgetting While RL Excels
joecole · x · 2026-08-12
A new paper provides a focused comparison between Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) in multi-task training.
- Multi-stage SFT collapses learning: When a language model is trained sequentially on math, code, logic, and science, new stages overwrite previously trained parameters, causing severe catastrophic forgetting. This explains why standard SFT parallelizes all datasets in mixed batches.
- RL performs better: In contrast, RL training generates near-orthogonal weight updates, preserving prior knowledge and resulting in superior overall performance.
Related event: Study Reveals Multi-stage SFT Causes Task Conflicts, Favoring RL(4 posts)→
More from Research
- Study Finds: Higher Reasoning Effort Makes LLMs Less Likely to Ask Clarifying Questions — xuanalogue · 2026-08-12
- Open Dataset Measures AI's Actual Impact on Accelerating Scientific Discovery — soumitrashukla9 · 2026-08-12
- Six Months of AI Auto-Research Tools, But No Clear Acceleration in Algorithmic Efficiency — soumitrashukla9 · 2026-08-12
- Research Reveals the Personality Evolution of the Grok Model Family — DevDminGod · 2026-08-12
- STACX: A Modular Infrastructure for End-to-End Agentic RL — daibond_alpha · 2026-08-12
- NeurIPS 2026 Workshop: World Models for High-Stakes Healthcare — yaringal · 2026-08-12