Reasoning Compression Without Hurting Capabilities
marcodsn · reddit · 2026-07-13
Core Conclusion
The author conducted a series of small-scale training experiments around "reasoning compression," aiming to reduce the reasoning tokens output by the model without destroying its reasoning capabilities.
Key Findings
- Full compression breaks greedy decoding: Uniformly compressing complete reasoning traces into a concise style caused the model to loop heavily on GSM8K, with accuracy nearly collapsing at temperature 0. However, accuracy recovered when the sampling temperature was increased, indicating the issue is "not knowing when to stop" rather than failing to learn.
- Segment-aware compression works: Preserving calculation and verification segments while only compressing/deleting narrative transitions not only maintained model performance on GSM8K but actually outperformed the uncompressed SFT baseline, while reducing reasoning tokens by about 1.7x.
- System prompt semantics are reshaped by training: For the original model, "think step by step" meant "think longer"; for the compression-trained model, it acted as an "efficiency trigger." However, if the prompt was included during training, the model became dependent on it. Thus, the author suggests a better approach: do not use it during training, but add it during deployment.
- Compression can act as a switch: After binding the compression style with an identity prompt, enabling the prompt resulted in shorter outputs; disabling it caused the model to "decompress" back to longer reasoning lengths while maintaining normal accuracy.
- Stronger teachers aren't always better: Using a larger model to segment/transform traces yielded cleaner-looking data, but the trained model performed worse than one trained directly on data near the original distribution.
- The "Code Tax" in the code domain: Qwen's code traces are inherently more calculation-heavy, making them almost impossible to compress. On coding tasks, compression training actually lengthened the thought process and hurt HumanEval/MBPP performance. Conversely, on Gemma, code was compressed more heavily, resulting in a notable improvement on HumanEval.
- Transferability to other model families: Applying the same process to gemma-4-12b-it resulted in a compressed model that achieved higher accuracy on GSM8K with fewer tokens, outperforming the original model across all sampling temperatures.
Scale and Resources
- Each training branch consisted of only about 322–648 samples
- Each branch required roughly 1.5k 3090 hours
- The author plans to scale up to larger datasets in the future
Related Resources
The author provided a comprehensive paper-style summary, along with the model/dataset collection and code.
More from Research
- OpenAI says long-horizon models need safety and alignment checks across full action sequences — rhiever · 2026-07-22
- A Reddit user proposes a consistency LoRA to keep anime and game scenes visually stable — ThirdWorldBoy21 · 2026-07-22
- Graph workload 854.graph500 enters SPEC CPU 2026 as a new CPU benchmark — Prof_DavidBader · 2026-07-22
- BlackboxNLP 2026 is recruiting extra reviewers after a high submission volume — hanjie_chen · 2026-07-22
- AWS shows self-distilled reasoning can preserve math and coding skills during SFT — AWS ML Blog · 2026-07-22
- UI2App shows screenshot fidelity still lags real interaction recovery — Grace Man Chen · 2026-07-22