WISD: Distill Training Signal from Weaker Quantized Copies of Reasoning Models
jet-ai · hf · 2026-10-31
A new paper proposes WISD (Weakening-Induced On-Policy Self-Distillation): it creates weakened versions of a large reasoning model via quantization or pruning, then uses the contrast between strong and weak versions to generate training signals—improving reasoning without privileged data such as ground-truth answers.
More from Research
- Tversky branch inside RNN cells: half the depth matches full xLSTM with 2x faster training — stanfordnlp · 2026-10-31
- AI pushes circuit lower bound from 3.1n to 3.5n as UCLA-backed challenge aims at P vs NP — soumitrashukla9 · 2026-10-31
- Qwenputer: a full RISC-V CPU inside Qwen3.5-4B weights, running C in forward passes — armand_ruiz · 2026-10-31
- What is work for? Cross-national survey of 8,234 adults on meaning in the AI age — soumitrashukla9 · 2026-10-31
- Paper: benchmark difficulty coverage is skewed, leaderboard conclusions need caution — Miserable-Dare5090 · 2026-10-31
- AISTATS 2027 workshop proposals due Nov 6 after one-week extension — qberthet · 2026-10-31