Prefix Sliding: 3x Faster LLM Reasoning Without Performance Loss
Muennighoff · x · 2026-08-27
Niklas Muennighoff et al. released "Prefix Sliding for efficient test-time scaling." Addressing the high compute cost of full attention in long-horizon reasoning, the study finds that intermediate tokens lose importance over time. The proposed Prefix Sliding method retains only the essential prefix (instructions/tools) and a sliding window of recent tokens. It achieves 3x speedup without training while maintaining performance, and can scale to over 100k tokens with RL fine-tuning. It outperforms summarization and vanilla sliding window baselines.
Related event: Prefix Sliding triples inference speed with no quality loss(4 posts)→
More from Research
- Hannah Fry: How OpenAI has triggered an existential crisis in mathematics — AnnaCiaunica · 2026-10-15
- Watching AI evaluation shift from offline benchmarks to online eval — vboykis · 2026-10-15
- StartLux-Decision debuts RL-trained calibrated decision models, 27B scores 72.8 — startlux-models · 2026-10-15
- A third of all math preprints in 9 months were written with AI, researcher says — IgorCarron · 2026-10-15
- AAAI/ACM AIES paper interviews 15 data scientists on why responsible AI practice is hard to sustain — ChinasaTOkolo · 2026-10-15
- Schmidt Sciences awards $10M to 19 AI humanities research teams — OdedRechavi · 2026-10-15