The Illustrated Recurrence: From Amari-Hopfield Nets to GPT-6 Astra
gklambauer · x · 2026-09-21
Researcher Günter Klambauer published a blog post, "The Illustrated Recurrence," mapping the lineage of recurrence in neural networks across 17 unrolled computation diagrams—from 1970s Amari-Hopfield nets to the rumored GPT-6 Astra.
Key points:
- Architectures sorted by what repeats and along which axis: no recurrence (where most production LLMs sit), recurrence along time, whole-network recurrence (output fed back as input), and recurrence along depth (shared weights, the heaviest exploiters of recurrence)
- A consistent visual notation: identical fill patterns mark shared weights, dashed frames with ×T/×L mark weight-sharing loops over time/depth
- The author notes recurrence keeps getting reinvented in ML, and flags that the GPT-6 Astra panel is based on an unconfirmed report
- Points readers to Schmidhuber's comprehensive overview for a deeper treatment
More from Models
- Unclear requirements mean burned tokens: an LLM is only as good as your context — romitheguru · 2026-09-21
- Solving Sudoku with graph coloring: an Astra demo worth a look — mariyaivasileva · 2026-09-21
- Jev sparks debate: training your own classifier won't beat Voyage's years of reranker tuning — zainhas · 2026-09-21
- Freebuff's $8/Month Ad-Supported Coding Sub Promises 15 Hours of DeepSeek Daily — gaganghotra_ · 2026-09-21
- Unverified claim: Grok 4.7 launching today with 500K context, multimodal, 4.6-level pricing — realsohamparekh · 2026-09-21
- Dev warns: if token subsidies end, 24/7 agent use gets priced out alongside local — BLUECOW009 · 2026-09-21