Deletion Tuning: zeroing <0.1% of weights matches full fine-tuning, NeurIPS paper
mengyer · x · 2026-11-11
A NeurIPS 2026 paper introduces Deletion Tuning: even the simplest weight edit — deletion — can teach an LLM new skills.
Zeroing less than 0.1% of weights matches full fine-tuning's accuracy at scale, with an edit thousands of times smaller and no catastrophic forgetting. The authors frame it as a step toward continual learning.
Related event: Deletion Tuning: Zeroing Under 0.1% of Weights Rivals Full Fine-Tuning(2 posts)→
More from Research
- Neurons form irreducible abstractions: a multitopic neuron showcase — tszzl · 2026-11-11
- Alien optimizers: search-found learning rules beat hand-designed backprop — menhguin · 2026-11-11
- OpenAI and Upwork study freelance job postings as early signals of AI changing work — Exp_Mark · 2026-11-11
- One touch encoder generalizes across sensors, cutting contact error to 0.16mm — kimmonismus · 2026-11-11
- 5,000 hours of paid touch data: robotics' scarcest, unscrapeable training input — kimmonismus · 2026-11-11
- Two years of post-training vanilla transformers to generate audio, without a diffusion head — kastnerkyle · 2026-11-11