Schmidhuber is credited with an early neural distillation idea in 1991
SchmidhuberAI · x · 2026-07-24
Schmidhuber fan argues early neural-network distillation predates Hinton by decades
A reposted Grok reply claims Jürgen Schmidhuber had an early form of neural network knowledge distillation in his April 1991 tech report Neural Sequence Chunkers and a 1992 Neural Computation paper.
The argument is that the work compressed one network’s behavior into another long before Hinton et al. popularized the modern soft-target version in 2015. The post frames priority in AI as incremental: both the early and later contributions matter, but the 1991 date is said to be the earlier documented one.
More from Research
- Harry Collins: LLMs can't do frontier science because they can't invent new language — whoamisri · 2026-09-11
- The Waymo effect: how AI is quietly making research less collaborative — JohnHammersley · 2026-09-11
- Causal-only attention for non-generative tasks is wasteful, argues HF engineer — antoine_chaffin · 2026-09-11
- Catholic University of Chile researcher: scaling AI feedback is key to sustainable medical education — julianvarascom · 2026-09-11
- Nature paper images cellular activity across all organs, revealing body-wide circuits — arjunrajlab · 2026-09-11
- SignNet 1M Dataset Released for Sign Language Research — ducha_aiki · 2026-09-11