Stanford's Prefix Sliding speeds up long reasoning by 3x
omarsar0 · x · 2026-08-31
Stanford researchers introduced Prefix Sliding, a new method to optimize test-time compute for long reasoning chains (CoT).
- Problem: Long reasoning traces require full attention, keeping the entire history in memory. This makes the hardest problems (which need the most thinking) the most expensive to run.
- Insight: Intermediate tokens steadily lose importance as the model continues reasoning.
- Solution: Prefix Sliding retains the prefix (instructions and available tools) and a sliding window of the last few thousand tokens, discarding everything in between. This caps total memory usage regardless of reasoning length.
- Results: Achieves a 3x speedup on existing models without any training, while matching full-attention performance.
More from Infra
- SK Hynix breaks ground on Indiana HBM plant, targeting HBM4e mass production in 2029 — Beth_Kindig · 2026-08-31
- Qwen 3.8 Flash Next runs at 3.5 tok/s on mid-range Android phone — dai_app · 2026-08-31
- The Boring Company's sales dilemma: 20x cost advantage but no sales team — PTrubey · 2026-08-31
- Running 182B Qwen on 4080: 8 tok/s via SSD offloading — Desperate-Data-3747 · 2026-08-31
- AI is transforming how energy systems are monitored, maintained and operated — ingliguori · 2026-08-31
- Uber Cuts AI Costs 52% While 10xing Usage via Agentic Workflow — alvelda · 2026-08-31