Schmidhuber says neural-network distillation was already published in 1991
SchmidhuberAI · x · 2026-07-23
Jürgen Schmidhuber argues that neural-network distillation is not a new idea and points to a 1991 publication of his own as the origin.
In the quoted discussion, he links the method to early work on transferring knowledge from one neural network to another and suggests later systems such as DeepSeek are using a principle that was already described decades ago.
Related event: Schmidhuber Claims Neural Network Distillation Dates Back to 1991(3 posts)→
More from Research
- Manual checks of 105 papers find most reproducibility failures come from missing code and mismatched results — ChenhaoTan · 2026-07-23
- A proxy gateway approach to stopping agent tool-call loops — bulleykebaal · 2026-07-23
- NEJM AI: Generative AI Intervention Effectively Mitigates College Student Depression — zakkohane · 2026-07-23
- Cohere to Host Talk on LLM Agent Reliability & Uncertainty Signals — Cohere_Labs · 2026-07-23
- Writer Study: Optimizing AI Harness Reduces Costs by 41% Without Losing Accuracy — bendee983 · 2026-07-23
- Reasoning traces from math breakthroughs may reveal how models think — benno_krojer · 2026-07-23