Schmidhuber says knowledge distillation was published in Europe in 1991
SchmidhuberAI · x · 2026-07-23
Jürgen Schmidhuber argues that knowledge distillation was already published in Europe in 1991, and later copied in the US and China.
In the quoted thread, he says the 2025 DeepSeek “Sputnik” moment reignited the debate over who invented distillation. He points to his 1991 work, where a teacher network’s knowledge could be compressed into a student network by imitation training.
The point of the post is historical and technical rather than product-focused: distillation is presented as an old, general principle for transferring knowledge between neural networks.
More from Research
- Causal-only attention for non-generative tasks is wasteful, argues HF engineer — antoine_chaffin · 2026-09-11
- Catholic University of Chile researcher: scaling AI feedback is key to sustainable medical education — julianvarascom · 2026-09-11
- Nature paper images cellular activity across all organs, revealing body-wide circuits — arjunrajlab · 2026-09-11
- SignNet 1M Dataset Released for Sign Language Research — ducha_aiki · 2026-09-11
- ECCV26 Oral: Flow Matching Enables Single-Stage Multi-View Point Cloud Registration — ducha_aiki · 2026-09-11
- InFlux++ Method Released — ducha_aiki · 2026-09-11