Triadic Linear Attention: A 3D Recurrent State Aims to Outscale Matrix-State Linear Attention
HanGuo97 · x · 2026-10-05
- Researcher osieberling proposes Triadic Linear Attention: linear attention is arguably the most naive RNN, yet it massively outperforms traditional RNNs by maintaining a matrix state. The new idea: use a (triadic) outer product of three vectors to maintain a three-dimensional state instead.
- The team, including DaviJin, worked on the systems side to make the much larger recurrent state efficient in practice, and reports that the extra capacity pays off at long context.
- Details are in the linked thread; results are early-stage and await community validation.
Related event: Triadic Linear Attention Extends Linear RNNs with 3D Tensor States(3 posts)→
More from Research
- ELLIS Institute Tübingen hires PIs: 6-year terms, no teaching, generous research budgets — HildeKuehne · 2026-10-05
- UVM Wins up to $38M to Build AI 'Digital Twins' for Critically Ill Patients, Could Cut ICU Stays 25% — HealthcareLdr · 2026-10-05
- SimuVerity Benchmark: Best Agent Scores Only 42.86 on Engineering-Grade Simulink Generation — Ruiqi Zhang · 2026-10-05
- MIT's Local Support Learning Fixes Catastrophic Forgetting in LLMs Without Old Data — MIT · 2026-10-05
- HelixWorld: Real-Time Audio-Visual World Model Runs at 24 FPS on a Single GPU — NoizAI · 2026-10-05
- BlockRank: sparse attention makes LLM in-context document ranking faster — dejanseo · 2026-10-05