SFT Conflicts, RL Coexists: Theoretical Analysis of Multi-Task LLM Training
CASIA · hf · 2026-08-10
A paper from CASIA provides a theoretical and empirical analysis comparing Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) in multi-task reasoning for LLMs.
The study reveals that SFT suffers from severe task conflicts in multi-stage training, whereas RL enables stable coexistence across diverse tasks. At the parameter level, RL induces sparse and approximately orthogonal updates across tasks.
Theoretically, interference in SFT is norm-limited, while in RL it is variance-limited. This small variance bound yields near-orthogonal optimization directions. Leveraging this insight, the authors propose Parallel-RL, a paradigm that decouples multi-task training to significantly improve efficiency and flexibility.
More from Research
- Study: LLMs Write More Correct Code in JS Than TS, Questioning Type Guardrails — hichaelmart · 2026-08-10
- 269 Hours of EEG Data: Non-Invasive Speech Decoding Hits 61.3% Accuracy — kaixhin · 2026-08-10
- Why Speculative Decoding Exploded: Tri Dao's Paper Fuels an Inference Revolution — Ok-River5924 · 2026-08-10
- Training Humanoid Robots in 3D Scans: Zero Real-World Fine-Tuning — lukas_m_ziegler · 2026-08-10
- MOSS-Transcribe-Diarize: Transcription and Diarization in One 0.9B Model — vanstriendaniel · 2026-08-10
- Harvard and MIT Open-Source MatrAIx: Simulating 8.3B Global Personas — SRSchmidgall · 2026-08-10