Training-free memory augmentation makes compressed CoT faster: +21.4 pts on GSM8K
rohanpaul_ai · x · 2026-09-01
A paper proposes a new way to make chain-of-thought cheaper: move reusable reasoning from generation into the prompt—instead of regenerating every reasoning step, provide the relevant reasoning pattern upfront so the model thinks shorter.
Memory-Augmented Compression is training-free: it distills solved examples into reusable reasoning memories, retrieves relevant ones per query, and injects them before compressed reasoning. This shifts work from slow autoregressive decoding to the more parallel prefill stage.
With Qwen2.5-7B, adding memory to Chain-of-Draft recovered 21.4 accuracy points on GSM8K and 28.0 on MATH, while latency stayed 1.49× and 1.14× faster than standard CoT.
More from Research
- Interpretability debate: one interpretable polysemantic neuron doesn't prove all are — aran_nayebi · 2026-10-16
- Optuna creator Akiba: when 2 hyperparameters cost $20M, what's left for BO/HPO — Stefania_druga · 2026-10-16
- AI will be superhuman at science regardless, and Europe can't opt out — teortaxesTex · 2026-10-16
- SERO augments long-horizon agentic RL with executable oracle continuations — Shanghai-AI-Laboratory · 2026-10-16
- Tsinghua and Amap propose OmniVerifier-C1, a graded verifier beating binary judges — Bowei Liu · 2026-10-16
- Alibaba open-sources MAI-Gym, an all-in-one GUI agent eval and RL training platform — Tongyi-MAI · 2026-10-16