'Grafting' paper: SFT on donor checkpoints + weight merging beats on-policy self-distillation
AdtRaghunathan · x · 2026-11-05
A CMU paper by Wu, Zhang & Raghunathan went viral as a 'huge training alpha drop'. SFT on new data learns useful signal but its naive application interferes with existing capabilities. Their grafting recipe: (1) learn the update on an earlier donor checkpoint—ideally before pretraining ends—then apply it to the post-trained model; (2) scale the update, a form of model merging; (3) mask sensitive update directions when data is far from the model. Off-policy merging beats OPSD for continual learning while avoiding reasoning collapse. A co-author adds this reflects a 'subtractive' version of their task arithmetic findings, betting it strengthens at scale; the quoting post notes OpenAI appears to attribute post-RL behaviors to specific SFT datasets by subtracting SFT weight deltas.
More from Research
- ExMR open-sources single-video full-body 3D motion capture with robot export — camenduru · 2026-11-05
- VA-TDM: Velocity Adversarial Distillation Sets SOTA for Few-Step Diffusion Models — Yihong Luo · 2026-11-05
- SleuthVLM beats Gemini 3.8 Flash on long-video QA without ever hearing audio — antgroup · 2026-11-05
- AnchorComplete reconstructs complete scene meshes from partial point clouds — Weirong Chen · 2026-11-05
- Neuroscientists debate using deep RL virtual animals as neuroscience research tools — neuroecology · 2026-11-05
- Berkeley team shows LLMs lose info via recursive self-summarization, fixes it with RL — berkeley_ai · 2026-11-05