How SFT and RL Restructure Reasoning Trajectories
tw_killian · x · 2026-07-10
This study mechanistically analyzes the roles of SFT and RL in standard post-training: SFT provides rich, compositional reasoning trajectories, while RL decomposes them into atomic skills and routing strategies.
Control experiments show this approach particularly boosts model performance on unseen combinations and varying reasoning depths, indicating that models can reuse familiar reasoning fragments to solve novel tasks.
More from Research
- Lab monkeys at $30-80K each: a 4-year breeding cycle is the real preclinical drug bottleneck — zebird0 · 2026-10-27
- OpenAI Foundation funds UNC tumor antigen project, hiring senior experimental scientist — iskander · 2026-10-27
- Steering attention heads toward honesty: up moral patienthood, down psychopathy — edelwax · 2026-10-27
- Are transcoders overkill? New paper argues plain neurons suffice for circuit analysis — burny_tech · 2026-10-27
- EMNLP paper GENIE: a fine-grained metric locating where LLM stories are novel — gregd_nlp · 2026-10-27
- Hashed n-gram value embeddings push GPT to 3.2115 loss, beating 30M track record — RobertTLange · 2026-10-27