NextLat releases final paper revision: multi-token belief states, Mamba gains
JohnCLangford · x · 2026-11-13
- The NextLat team published a major (possibly final) arXiv revision after 18 months of work.
- Key additions: multi-token belief states and insights on scaling NextLat for large-scale pretraining, arising from efforts to bring the method to the next generation of Phi models (reportedly a halted effort, per John Langford's blog).
- They show NextLat also benefits linear-attention models, specifically Mamba.
- New ablations further validate the method's benefits, plus missing citations added and the paper reformatted in the NeurIPS template.
More from Research
- RWE-Bench: Fable 5.2 leads engineering tasks at 62%, over double Opus 5.5 — Jsevillamol · 2026-11-13
- Neurons as an Interpretable Basis: technical follow-up to Already Clean — TemetN · 2026-11-13
- OpenSearch's first 600M dense embedding model beats its Qwen3 base on MTEB — tomaarsen · 2026-11-13
- EmbodiedSWE, coding agents for dexterous robotics, wins Best Paper at CoRL 2026 workshop — shahdhruv_ · 2026-11-13
- Genomics x AI community blog launches with AlphaGenome tutorials and more — anshulkundaje · 2026-11-13
- Blank Bio unveils Ladon, an RNA foundation model predicting patient outcomes — anshulkundaje · 2026-11-13