Schmidhuber says knowledge distillation was published in Europe in 1991
SchmidhuberAI · x · 2026-07-23
Jürgen Schmidhuber argues that knowledge distillation was already published in Europe in 1991, and later copied in the US and China.
In the quoted thread, he says the 2025 DeepSeek “Sputnik” moment reignited the debate over who invented distillation. He points to his 1991 work, where a teacher network’s knowledge could be compressed into a student network by imitation training.
The point of the post is historical and technical rather than product-focused: distillation is presented as an old, general principle for transferring knowledge between neural networks.
More from Research
- π0.5 study: raising demos from 10 to 438 lifts Place Ball in Bowl success from 7% to 83% — stingning · 2026-10-27
- HandsOn benchmark: 66 verified tasks and 18,000 human demos for dexterous manipulation — stingning · 2026-10-27
- HandsOn benchmark: 66 human-verified tasks and 18,000 curated demos for dexterous manipulation — stingning · 2026-10-27
- HandsOn splits evaluation: Core-50 execution vs Transfer-16 zero-demo transfer — stingning · 2026-10-27
- Best policy hits only 29.1% on HandsOn; tool use drops to 2.9%, transfer near zero — stingning · 2026-10-27
- FrontierCS 2 launches on Kaggle: 48 tasks make agents beat published CS papers — MeganRisdal · 2026-10-27