LIFT reintroduces recurrence into Transformers without losing parallel pretraining, halving data needs
megamor2 · x · 2026-10-30
A new paper introduces LIFT, a language model that passes a latent state from its last layer back into its first layer at every step — bringing recurrence back to Transformers while still pretraining fully in parallel. The authors claim a Transformer needs 2x the training data to catch up with LIFT at equal quality.
Related event: LIFT: New Architecture Feeds Latent States Back Into Transformers(2 posts)→
More from Research
- Researcher: capability transfer hits a ceiling when training against less human-like models — saurabh_shah2 · 2026-10-31
- Compaction study: agent constraints vanish 6x more often than task goals in long sessions — maksym_andr · 2026-10-31
- CMU's Gym-Anything uses coding agents to auto-build 200 robot RL environments; best model scores 0% — AkariAsai · 2026-10-31
- Cherimoya ChIP-seq models ship as-is: bad underlying data means bad predictions — jmschreiber91 · 2026-10-31
- When chromatin models disagree: null accessibility shift but a disrupted CEBPG binding site — jmschreiber91 · 2026-10-31
- Whole-Body Dexterity turns the humanoid body into one large dexterous manipulator — chris_j_paxton · 2026-10-31