Negative Self-Distillation improves LLM reasoning by avoiding flawed reasoning paths
Rongcan Pei · hf · 2026-09-11
A new method, Negative Self-Distillation, improves LLM reasoning by pushing models away from self-generated flawed reasoning paths instead of imitating positive samples.
- Relies on a dynamic gating mechanism to protect the model's general linguistic capabilities during training
- Offers a training paradigm complementary to positive SFT/RLHF: learning from avoiding mistakes rather than imitating correct answers
More from Research
- Programmable Cellular Automata: CA rules as readable code for explainability — Amidos2006 · 2026-09-11
- GEVIBench launches as a comprehensive benchmark for comparing voltage indicators — drmichaellevin · 2026-09-11
- Gaussian Light Transport: 13D Gaussian Mixtures Speed Up Global Illumination — ssh4net · 2026-09-11
- Fortnow: P vs NP beyond AI's reach, but NP vs L separations could fall — fortnow · 2026-09-11
- MaP-WAM tackles non-Markovian robot manipulation with memory-grounded planning — Sizhe Zhao · 2026-09-11
- DeepMind-led paper makes design docs the source of truth, code disposable — SMART regenerates in 1.5-3h for ~$100 — Roger_M_Taylor · 2026-09-11