Prefix Sliding: Discard Intermediate Reasoning Tokens to Cut Test-Time Compute, Training-Free
burkov · x · 2026-09-08
A paper introduces Prefix Sliding, addressing test-time scaling's costs: full attention keeps every prior token, so compute grows linearly with length, plus distraction by irrelevant tokens, repetitive loops, and lost information. The question: can most intermediate reasoning tokens be discarded without harming results?
- The method retains only a fixed prefix of system instructions/task prompt plus a sliding window of the most recent few thousand tokens; it is training-free and works on existing models, including inside RL rollouts.
- Experiments use Qwen3-1.7B on GPQA, MATH500, and AIME25, averaging 64 runs per setting, compared against full attention and three bounded-cost baselines.
More from Models
- OCR is not solved: SOTA model reads one 1881 line, outputs 16,421 dots — vanstriendaniel · 2026-10-26
- Gemini 4 Argon Lands 6th on RSI-Inference Bench, First Non-OA/Anthropic Model to Pass Day-0 Tasks — DynamicWebPaige · 2026-10-26
- Android Codex bug misreports model tiers: Astra Max shows as Astra Medium — SIGKITTEN · 2026-10-26
- Zvi digests Gemini 4 Argon cards: 217-page safety report, 0.7% injection success — TheZvi · 2026-10-26
- Nemotron 3.5 Super VL tested on 300 real bills: reasoning lifts accuracy 91% to 98% — Arindam_1729 · 2026-10-26
- Nanbeige4.3-33B benchmarked: 59 tok/s single stream, ~1,060 tok/s across 64 concurrent runs — MaziyarPanahi · 2026-10-26