Schmidhuber says neural-network distillation was already published in 1991
SchmidhuberAI · x · 2026-07-23
Jürgen Schmidhuber argues that neural-network distillation is not a new idea and points to a 1991 publication of his own as the origin.
In the quoted discussion, he links the method to early work on transferring knowledge from one neural network to another and suggests later systems such as DeepSeek are using a principle that was already described decades ago.
More from Research
- Causal-only attention for non-generative tasks is wasteful, argues HF engineer — antoine_chaffin · 2026-09-11
- Catholic University of Chile researcher: scaling AI feedback is key to sustainable medical education — julianvarascom · 2026-09-11
- Nature paper images cellular activity across all organs, revealing body-wide circuits — arjunrajlab · 2026-09-11
- SignNet 1M Dataset Released for Sign Language Research — ducha_aiki · 2026-09-11
- ECCV26 Oral: Flow Matching Enables Single-Stage Multi-View Point Cloud Registration — ducha_aiki · 2026-09-11
- InFlux++ Method Released — ducha_aiki · 2026-09-11