Prefix Sliding: Discard Intermediate Reasoning Tokens to Cut Test-Time Compute, Training-Free
burkov · x · 2026-09-08
A paper introduces Prefix Sliding, addressing test-time scaling's costs: full attention keeps every prior token, so compute grows linearly with length, plus distraction by irrelevant tokens, repetitive loops, and lost information. The question: can most intermediate reasoning tokens be discarded without harming results?
- The method retains only a fixed prefix of system instructions/task prompt plus a sliding window of the most recent few thousand tokens; it is training-free and works on existing models, including inside RL rollouts.
- Experiments use Qwen3-1.7B on GPQA, MATH500, and AIME25, averaging 64 runs per setting, compared against full attention and three bounded-cost baselines.
More from Models
- davinci-002 shuts down Sept 28: dev recreates the original GPT experience as a farewell — cephaloform · 2026-09-08
- Founder's model ranking: ChatGPT underrated, Gemini 'knocked flat', Claude overhyped — firstadopter · 2026-09-08
- Engineer Joshua Saxe: Astra useful but far from AGI hype — a 'slot machine' AI with no memory of intent — joshua_saxe · 2026-09-08
- GPT-6 Astra crushes Claude in Blender tasks — the gap is training data, not intelligence — bookwormengr · 2026-09-08
- Muse Spark 1.3 claims benchmark wins over GPT-5.6, skeptics doubt real-world use — teortaxesTex · 2026-09-08
- IFM's K2-Horizon-MoVA-36B-A4B draws praise as "surprisingly solid" — rupspace · 2026-09-08