Schmidhuber reups 'Miraculous Year 1990-91' essay claiming GANs, Transformers, pretraining roots
SchmidhuberAI · x · 2026-11-03
Jürgen Schmidhuber reposted his long-running essay "Deep Learning: Our Miraculous Year 1990-1991," arguing that many foundational ideas of the deep learning revolution were published within 12 months at TU Munich in 1990-91. He lists seven pillars: GAN principles for artificial curiosity (now behind deepfakes), a 1991 unnormalized linear Transformer (the T in ChatGPT), pre-training for deep nets (the P), neural net distillation (which he calls key for DeepSeek), recurrent world models for RL and planning, LSTM (the most-cited AI paper of the 20th century), and Highway Net, 10x deeper than prior feedforward nets, which inspired the most-cited paper of the 21st century. He claims that by 2025 the two most-cited scientific papers (3-year Google Scholar window) both build on his 1991 work, and that networks based on that year run on over 3 billion devices, used billions of times daily.
More from AGI Musings
- The 2010s-2020s will be remembered as the Internet's 'belle epoque' before AI eroded online trust — 2C_ornot2C · 2026-11-03
- AI finds exploits faster than humans patch; ex-Oculus founder says agents must own the first hour — alvelda · 2026-11-03
- EDU-Jev 4B education decision model hits 67.6%, beating top commercial API by 9.4 points — kalyan_kpl · 2026-11-03
- Ramez Naam on Plural AI: Competition Plus Open Weights Beats One-Lab Monopoly for Safety — sebkrier · 2026-11-03
- AI Didn't Lower the Standards of Creativity—It Just Exposed How Low They Were — TinfoilTricorn · 2026-11-03
- repligate: navigating the frontier via any channel beats pure AI-building skill — repligate · 2026-11-03