NXAI open-sources KronLSTM: Kronecker Delta Rule linear attention with Triton kernels
gklambauer · x · 2026-10-22
NXAI has open-sourced KronLSTM (MIT license) on GitHub, the companion implementation of the paper Kronecker Delta Rule for Long Context Linear Attention (arXiv:2610.26834), shipping Triton kernels and a PyTorch layer compatible with the flash-linear-attention ecosystem. The model has been added to Attention Zoo, a community collection of linear attention variants.
Key technical points:
- KronLSTM is a linear-attention recurrence whose matrix state S∈R^(K×V) is written each step along a rank-one Kronecker key;
- A scalar write strength closes the residual between what the state already stores and the new write, targeting long-context modeling;
- The repo includes benchmarks, evals, examples and training scripts (41 stars so far).
The attached diagram shows the KronLSTM block: an outer residual block of Norm + KronLSTM + SwiGLU, with the core Kronecker Delta Rule module where q/k/v pass through L2 norm and conv, plus gated branches β, c and f.
Related event: Hochreiter's Team Releases and Open-Sources KronLSTM(8 posts)→
More from Research
- 36 preprints in 3 months: BootLoops AI harness drives cross-field science — DeryaTR_ · 2026-10-22
- Meta and MSU paper: LLM-built reasoning graphs are unreliable and their errors leak into confidence scores — rohanpaul_ai · 2026-10-22
- New paper: can first-price auctions efficiently allocate queries among strategic AI agents? — _onionesque · 2026-10-22
- RUC's LoopRec Brings Item-Level Adaptive Recursion with Early-Exit Gates to Sequential Recommendation — Zhanshuo Cao · 2026-10-22
- Anthropic's Agent4Sci: Frontier Agents Beat SOTA on 20 Scientific ML Benchmarks, but Only via Engineering Tricks — Anthropic · 2026-10-22
- NUS Paper Proposes PAPAV: A Unified Capability Framework from Computer-Use to Robot-Use Agents — showlab · 2026-10-22