Schmidhuber says neural-network distillation was published in 1991
SchmidhuberAI · x · 2026-07-23
Jürgen Schmidhuber argues that neural-network distillation was already published in 1991, and that later work merely repackaged the idea.
He frames the discussion by pointing to a 2025 DeepSeek-related debate about who invented distillation, then says the original paper described a general teacher-student compression principle: a student network can learn by imitating a teacher’s behavior, thereby compressing knowledge into a smaller model.
Related event: Schmidhuber Claims Neural Network Distillation Dates Back to 1991(3 posts)→
More from Research
- Manual checks of 105 papers find most reproducibility failures come from missing code and mismatched results — ChenhaoTan · 2026-07-23
- A proxy gateway approach to stopping agent tool-call loops — bulleykebaal · 2026-07-23
- NEJM AI: Generative AI Intervention Effectively Mitigates College Student Depression — zakkohane · 2026-07-23
- Cohere to Host Talk on LLM Agent Reliability & Uncertainty Signals — Cohere_Labs · 2026-07-23
- Writer Study: Optimizing AI Harness Reduces Costs by 41% Without Losing Accuracy — bendee983 · 2026-07-23
- Reasoning traces from math breakthroughs may reveal how models think — benno_krojer · 2026-07-23