Dynamic weight grafting localizes how LLMs store facts learned during finetuning
ChenhaoTan · x · 2026-10-06
An arXiv paper proposes dynamic weight grafting: selectively grafting weight subsets from a finetuned model onto the pretrained model to localize where newly finetuned facts (new movies, new pope, etc.) live. The study finds two distinct retrieval pathways: 'enriching' the residual stream with relation information while processing entity tokens, and 'recalling' the fact at the final token position. Both pathways are sometimes jointly needed, otherwise either alone suffices, and the recall pathway is localized to specific components. The tweet also notes a related trick: finetuning the base model with next-token prediction on synthetic documents and applying that weight update directly to the post-trained model works.
More from Research
- COLM 2026 Paper: Reasoning Fine-Tuning Induces Persistent Latent Policy States in LLMs — hunarbatra · 2026-10-06
- Mathematician Claims AI Has Proven the KLS Conjecture — michaelchchoi · 2026-10-06
- hardmaru Explains Twitter's Logo Via Schmidhuber's Formal Theory of Fun and Creativity — hardmaru · 2026-10-06
- ViDiHand: Video Diffusion Models Prove Surprisingly Good at 4D Hand Motion Reconstruction — ccloy · 2026-10-06
- 1-in-4 to 1-in-3 dissertations now written with AI, replicated study finds — skdh · 2026-10-06
- Researcher maps AI Village data to Ising and subcritical Hawkes systems — ctjlewis · 2026-10-06