Tversky branch inside RNN cells: half the depth matches full xLSTM with 2x faster training
stanfordnlp · x · 2026-10-31
An experiment shared via Stanford NLP adds a Tversky similarity branch to the mLSTM cell used in Vision-LSTM and xLSTM.
- At equal parameter counts, half the depth matched or beat the full-depth baseline
- 20% fewer FLOPs and up to 2x faster training
- The idea embeds Tversky similarity (an asymmetric generalization of set similarity) directly inside the recurrent cell
Full write-up available in the linked article.
More from Research
- CMU paper unifies pretraining, SFT and RL into one likelihood-based training algorithm — rohanpaul_ai · 2026-10-31
- 48-page deep dive: why modern LLM RL is secretly off-policy and 14 ways to fix it — cwolferesearch · 2026-10-31
- Google Health: AI Tools Aim to Support Psychiatrists Where Conversation Is Diagnosis — SRSchmidgall · 2026-10-31
- Benchmarking 'scoping': five ways to confine LLMs to a bounded operational domain — burkov · 2026-10-31
- Meta paper: looping transformer weights adds depth without more parameters, up to 32 loops — iScienceLuvr · 2026-10-31
- Simple neg-log-odds loss swap improves every training attribution method tested — davidbau · 2026-10-31