Prefix Sliding: 3x Faster LLM Reasoning Without Performance Loss
Muennighoff · x · 2026-08-27
Niklas Muennighoff et al. released "Prefix Sliding for efficient test-time scaling." Addressing the high compute cost of full attention in long-horizon reasoning, the study finds that intermediate tokens lose importance over time. The proposed Prefix Sliding method retains only the essential prefix (instructions/tools) and a sliding window of recent tokens. It achieves 3x speedup without training while maintaining performance, and can scale to over 100k tokens with RL fine-tuning. It outperforms summarization and vanilla sliding window baselines.
Related event: Prefix Sliding triples inference speed with no quality loss(4 posts)→
More from Research
- Qwen3.8-Next paper: matches 397B predecessor with 1/9 the training FLOPs — NielsRogge · 2026-08-28
- How 'Making It Worse' Made $20M: The Tech Behind Bodycam — aakashgupta · 2026-08-28
- NLP sarcasm detection challenge: Who is building SarcasmBench? — paul_cal · 2026-08-28
- PILOT Enables Live Self-Improvement for Long-Horizon Agents via Skill Distillation — PolyUHK · 2026-08-28
- Skild vs GEN: Both Call It In-Context Learning, Very Different Paths — gan_chuang · 2026-08-28
- Paper analyzes thermal tuning overhead in optical interconnects for MoE training — jwt0625 · 2026-08-28