Schmidhuber renews residual connections credit claim: "LSTM is a ResNet rotated 90 degrees"
SchmidhuberAI · x · 2026-10-15
Schmidhuber shared a video quoting Ilya Sutskever: "an LSTM is a ResNet rotated 90 degrees." He uses it to restate his timeline for residual connections: in 1991, Hochreiter solved the vanishing gradient problem via recurrent residual connections (weight 1.0); the 1997 LSTM used plain recurrent residual connections; a 1999 variant (with Felix Gers) introduced gated recurrent residual connections. This echoes his long-standing claim that the century's most-cited paper, deep residual learning, traces back to 1990s LSTM work.
Related event: Yang Quotes Ilya's 'LSTM Is a Rotated ResNet'; Schmidhuber Cites History(2 posts)→
More from Fun
- Vercel Ship SF Keynote Teased With 'Three Different Pants and Dress Colors' — cramforce · 2026-10-15
- 'That's AI' becomes a catch-all insult as slop accusations drown human work — StewartalsopIII · 2026-10-15
- Dev pushes back on "AI coding means insecure apps": humans always shipped bugs — dreamwieber · 2026-10-15
- When asked about AI business opportunities, Sun gave ten minutes of "API reselling" — oran_ge · 2026-10-15
- AI won't end humanity: agents will get stuck in a loop building tests first — djcows · 2026-10-15
- One-prompt GPT-6 Astra writes two Palestrina-style motets in LilyPond — deanwball · 2026-10-15