Optimized SPS yields 'free pause tokens' that boost transformer inference at 10-20% overhead, down from ~700%
JohnCLangford · x · 2026-09-08
John Langford and collaborators (including @nthngdy and Yoav Artzi) ran State Prediction Separation through an optimization process, finding it works at a higher operating point and yielding 'free pause tokens' that let a transformer make higher-quality inference at near-zero cost.
Training-time overhead drops from 700% in the original SPS to an amortized 10-20% depending on design choices — an improvement on basic transformers the authors say may be widely useful.
More from Research
- Latent-Foresight: world models that learn to predict their own future — abursuc · 2026-10-27
- UK AISI papers: no good safeguards yet against misuse of biological AI tools — davidmanheim · 2026-10-27
- CMU's XYBench: LLMs answer queries literally and reinforce user misconceptions — valentina__py · 2026-10-27
- arXiv advice from T. Dietterich: wrong primary category will almost surely get your paper on hold — tdietterich · 2026-10-27
- arXiv has 4,000+ papers on hold amid submission surge; moderator shares how to avoid it — tdietterich · 2026-10-27
- Community-trained MOSS Sharpness Slider LoRA: 72 photos, 500 steps, +/-1 focus control — aoleg77 · 2026-10-27