Schmidhuber Reiterates 1991 Fast Weight Programmers Were the First Transformers

On November 3, Jürgen Schmidhuber used his official X account to repost a series of items, reasserting his long-standing claim on deep learning history: many of the fundamental ideas behind the deep learning revolution were born within a 12-month span in 1990-91 at his TU Munich lab.

Confirmed

• Schmidhuber reposted his classic long-form piece, "Deep Learning: Our Miraculous Year 1990-1991," arguing that seven foundational pillars — including GANs and Transformer pre-training — were laid down in that period
• He reiterated that his 1991 Fast Weight Programmers research contained an unnormalized linear Transformer, i.e., the earliest prototype of the Transformer
• Responding to community discussion, he called current linear attention work "old hat long ago," saying these supposedly new things are exactly what neural fast weight programmers did in the 1990s
• User @arthurcolle noticed SchmidhuberAI (the official AI account) dismissively commenting "this is an old trick" on a new work, playing on the classic AI community meme: "You think you invented something new? Schmidhuber did it in 1991"

Why it matters

This is yet another public restatement of a position Schmidhuber has held for years: that the core technologies of modern deep learning — especially the Transformer and linear attention — stem from his work around 1991. While the community often jokes this is a recurring routine, it keeps sparking discussion about AI history and the attribution of academic contributions

2026-11-03 ~ 2026-11-03 · 5 related posts

• Schmidhuber reups 'Miraculous Year 1990-91' essay claiming GANs, Transformers, pretraining roots — SchmidhuberAI · 2026-11-03
• Schmidhuber revives his 1991 fast weight programmers: the first linear Transformer variant — SchmidhuberAI · 2026-11-03
• The classic meme strikes again: Schmidhuber did it in 1991 — arthurcolle · 2026-11-03

2 near-duplicate retellings: SchmidhuberAI · zainhas