Schmidhuber: Yale's fast weight programming course reprises his 1990s linear Transformer work
SchmidhuberAI · x · 2026-11-06
Yale instructor Kazuki Irie's CPSC 7770 course covered weight-space programming: gradient descent as a weight-space programmer, fast weight programming, metalearning (GD writes learning algorithms), and self-referential weight matrices with RSI. Schmidhuber quote-Tweeted that it's an old hat — neural fast weight programmers did exactly this in the 1990s, including the 1991 unnormalised linear Transformer. Course slides detail the slow net/fast net architecture, collapsing meta-levels into a self-referential loop, and metalearning as differentiable learning algorithms.
More from Research
- Frank Nielsen's taxonomy maps every distance and divergence from Euclidean to quantum geometry — pickover · 2026-11-06
- Anthropic report's key estimates are "guesses" — framing hides massive uncertainty, researcher says — daveholtz · 2026-11-06
- Project AURORA: a big-bet AI Cancer Co-scientist initiative launches with CRUK and NCI backing — zakkohane · 2026-11-06
- PNAS paper: kids explore farther than RL agents, driven by empowerment not info gain — _akpiper · 2026-11-06
- Google open-sources Agile Deliberation, an AI framework that turns fuzzy classification rules into explicit ones — aigclink · 2026-11-06
- ETH RSL's ACP humanoid: reference-free contact planning runs onboard at 50Hz — ChongZzZhang · 2026-11-06