Grafting: model merging after off-policy SFT beats on-policy self-distillation for continual learning
AdtRaghunathan · x · 2026-11-03
AdtRaghunathan's lab releases the first version of "grafting", challenging the view that continual learning must be on-policy:
- Common wisdom: off-policy learning causes forgetting; on-policy self-distillation with thinking models is messy
- Key finding: simply merging models after off-policy SFT goes a long way toward fixing forgetting
- Extra trick: learn the update on an earlier checkpoint and "graft" it onto the final model
- Results: grafting beats on-policy self-distillation and substantially improves continual learning on hard math reasoning and knowledge injection
Framed as a combination of the bitter lesson (let off-policy data speak for itself) and appropriate inductive biases, part of the lab's weekly AI safety / continual learning releases.
More from Research
- Recursive GPT-5-Mini calls often beat a single GPT-5 call for less money — lateinteraction · 2026-11-03
- Same Agent Reimplements Liquid in Rust vs C: Rust Uses 42% Fewer Tokens, C Runs 1.7x Faster — antirez · 2026-11-03
- After viral 'fly plays Beat Saber', researchers call for shared whole-brain emulation benchmark — rishabh16_ · 2026-11-03
- Researchers uncover 120 Google search twiddlers, including AI Mode's fan-out plan — VeryWellVersed · 2026-11-03
- Qwenputer compiles a full RISC-V CPU into Qwen3.5-4B weights, bit-exact, even running another LLM — udmrzn · 2026-11-03
- Astra IDs Breakout, Pong, Pac-Man and Freeway from raw model weights alone — burny_tech · 2026-11-03