Linear RNNs can recognize a periodic language under exact arithmetic
lambdaviking · x · 2026-11-04
A technical exchange on the expressive power of linear RNNs. The author notes that assuming a specific evaluation order feels wrong for architectures where the order can change between training and inference, and that a more general understanding across evaluation orders would be interesting. The reply clarifies: for GDN that holds, but for general linear RNNs there is a construction of a periodic language that works under exact associative arithmetic.
More from Research
- ipSAE extends to ligands: fixing poor ipTM scoring in AF3 and Boltz-2 cofolding models — sokrypton · 2026-11-05
- CMU's Visual Token Routing cuts VLM agent visual tokens up to 5.8x with no accuracy loss — wellecks · 2026-11-05
- TLAPS-Bench and agentic AI surface a read-consistency bug in SlateDB's WAL spec — tianyin_xu · 2026-11-05
- CellType claims 27B biology model beats all 12 frontier models at predicting cell behavior — alifcoder · 2026-11-05
- No-Prerequisite Information Geometry Tutorial Released With Video and PDF Notes — FrnkNlsn · 2026-11-05
- structRFM: structure-guided RNA foundation model trained on 21M pairs lands in Nature — EricTopol · 2026-11-05