Schmidhuber is credited with an early neural distillation idea in 1991
SchmidhuberAI · x · 2026-07-24
Schmidhuber fan argues early neural-network distillation predates Hinton by decades
A reposted Grok reply claims Jürgen Schmidhuber had an early form of neural network knowledge distillation in his April 1991 tech report Neural Sequence Chunkers and a 1992 Neural Computation paper.
The argument is that the work compressed one network’s behavior into another long before Hinton et al. popularized the modern soft-target version in 2015. The post frames priority in AI as incremental: both the early and later contributions matter, but the 1991 date is said to be the earlier documented one.
More from Research
- rauchg reproduces OpenAI's covert CPU-channel finding: chat across Firecracker microVMs with no network — ccerrato147 · 2026-10-27
- Microsoft Research: ForkPilot cuts long-horizon agent compute by up to 59% — EliasEskin · 2026-10-27
- Microsoft's TeleTune evolves agent skills from offline user telemetry — EliasEskin · 2026-10-27
- FlexGlue: sparse-to-dense matching that beats dense accuracy at a fraction of the cost — zhenjun_zhao · 2026-10-27
- Mundo turns a few room photos into an explorable, simulatable 3D world — at 1.5 FPS — mark_k · 2026-10-27
- Mathematics as an experimental science: Marc Lelarge to talk LLM-driven math experiments at ENS — marc_lelarge · 2026-10-27