KronLSTM: xLSTM team's Kronecker Delta Rule takes aim at Transformers
gklambauer · x · 2026-10-20
Sepp Hochreiter's group released KronLSTM (arXiv:2610.26834), a linear attention architecture based on a novel Kronecker Delta Rule for associative matrix memories.
- Key idea: instead of correcting the whole vector stored at a key (as in Delta Rule), it corrects a single scalar — how strongly a key is bound to a value — performing minimally invasive memory revision along one direction in matrix space, reducing interference with stored memories.
- Theory: memory dynamics form a Markov chain converging to a steady-state distribution, with uniformly bounded expected memory norm across the whole sequence — no extra normalization or stabilization needed.
- Results: competitive with or better than (Gated) DeltaNet-family linear attention models, especially strong on long-context benchmarks.
- Open-source code and models; extends the xLSTM family as a candidate alternative to Transformers for long-context LLMs.
Related event: NX-AI Unveils KronLSTM with Kronecker Delta Rule for Linear Attention(2 posts)→
More from Research
- New paper: let models reward hack in RL, then disable the parameters at deployment — nptacek · 2026-10-20
- Victor Taelin: smart enough compilers will understand and swap your algorithms — bronzeagepapi · 2026-10-20
- Galbot proposes observational scaling law for physical recursive self-improvement in VLM agents — Galbot · 2026-10-20
- ActionRoPE encodes action as world coordinates for spatially consistent game world models — Zhiyang Deng · 2026-10-20
- UniAVC unifies audio-video generation via progressive training and asymmetric noise scheduling — Kien T. Pham · 2026-10-20
- AHE paper auto-evolves coding-agent harnesses with decoupled files, lifting Terminal-Bench 2 to 77% — morgymcg · 2026-10-20