Microsoft's Free Pause Tokens Matches 50% More Token Training With Just 14% Extra Time
rohanpaul_ai · x · 2026-09-07
A Microsoft + Cornell paper, "Free Pause Tokens," gives prediction its own extra computation without adding tokens, growing the KV cache, or adding a decode step. It matched a standard model trained on 50% more tokens while adding only 14% training time and 1% inference latency.
Key findings:
- The pause needn't run for the whole training: switching it on after 42.5% of training kept 94% of the full quality gain at 1.33× wall-clock time.
- At equal node-hours, phased versions still beat standard training.
Recommendation: train normally for most of the run, then add the extra prediction computation near the end.
Related event: Microsoft and Cornell Propose Free Pause Tokens for Better Predictions(2 posts)→
More from Infra
- FP16 vs FP32 matmul on RTX 4060: 3.5x speedup at large sizes, slower at 128x128 — cneuralnetwork · 2026-10-23
- BaseQuant V1 quants for agentic Qwen3.8-27B hit 17-34% lower KL than Unsloth — nightowlsleeping · 2026-10-23
- PhD students in some countries get B200s on demand, unlike their US peers — HildeKuehne · 2026-10-23
- Nvidia's Berlin GTC spotlights humanoid robots and Vera Rubin AI supercomputing — emmanuelvivier · 2026-10-23
- Lenovo adds 5 AI Library use cases, claims 1-week deployment and up to 70% lower token cost — shashib · 2026-10-23
- Linux Kernel Module Unlocks Half-Speed Tensor Cores on RTX 3090, Boosting BF16 to 1.9x — UltramanQuar · 2026-10-23