Deletion Tuning: Zeroing Under 0.1% of Weights Rivals Full Fine-Tuning
A NeurIPS paper introduces Deletion Tuning, which learns a binary mask on the gate projection and zeroes out under 0.1% of weights, matching full fine-tuning on three post-training tasks without any other modifications.
2026-11-11 ~ 2026-11-11 · 2 related posts
- Deletion Tuning: zeroing <0.1% of weights matches full fine-tuning, NeurIPS paper — mengyer · 2026-11-11
- Deletion Tuning: Zeroing 0.05% of Weights Matches Full Fine-Tuning With Less Forgetting — Thrumpwart · 2026-11-11