Triadic Linear Attention Extends Linear RNNs with 3D Tensor States

Researchers including MIT's osieberling proposed Triadic Linear Attention, which maintains a 3D tensor state via the outer product of three vectors, extending linear attention beyond matrix states to boost long-context memory capacity.

2026-10-05 ~ 2026-10-05 · 3 related posts

• Triadic Linear Attention: extending matrix-state RNNs to a 3D tensor state — ChengleiSi · 2026-10-05
• Triadic Linear Attention: A 3D Recurrent State Aims to Outscale Matrix-State Linear Attention — HanGuo97 · 2026-10-05
• MIT's Triadic Linear Attention Expands RNN State to 3D Tensors for Better Long-Context Recall — Massachusetts-Institute-of-Technology · 2026-10-05