Schmidhuber says distillation was published in 1991 and later copied
tom_doerr · x · 2026-07-23
Jürgen Schmidhuber says a key fun fact about distillation is that it was published in 1991 and later copied by others.
The attached screenshot shows the broader context: he is pointing back to an early paper lineage and contrasting it with later claims in the modern AI boom. The post is mainly a historical reminder about how the technique predates many recent discussions.
More from Research
- Lab monkeys at $30-80K each: a 4-year breeding cycle is the real preclinical drug bottleneck — zebird0 · 2026-10-27
- OpenAI Foundation funds UNC tumor antigen project, hiring senior experimental scientist — iskander · 2026-10-27
- Steering attention heads toward honesty: up moral patienthood, down psychopathy — edelwax · 2026-10-27
- Are transcoders overkill? New paper argues plain neurons suffice for circuit analysis — burny_tech · 2026-10-27
- EMNLP paper GENIE: a fine-grained metric locating where LLM stories are novel — gregd_nlp · 2026-10-27
- Hashed n-gram value embeddings push GPT to 3.2115 loss, beating 30M track record — RobertTLange · 2026-10-27