Training-free memory augmentation makes compressed CoT faster: +21.4 pts on GSM8K
rohanpaul_ai · x · 2026-09-01
A paper proposes a new way to make chain-of-thought cheaper: move reusable reasoning from generation into the prompt—instead of regenerating every reasoning step, provide the relevant reasoning pattern upfront so the model thinks shorter.
Memory-Augmented Compression is training-free: it distills solved examples into reusable reasoning memories, retrieves relevant ones per query, and injects them before compressed reasoning. This shifts work from slow autoregressive decoding to the more parallel prefill stage.
With Qwen2.5-7B, adding memory to Chain-of-Draft recovered 21.4 accuracy points on GSM8K and 28.0 on MATH, while latency stayed 1.49× and 1.14× faster than standard CoT.
Related event: Training-Free Memory Augmentation Makes Chain-of-Thought Cheaper and Faster(2 posts)→
More from Research
- CMU Researchers Use AI to Explore Alzheimer's DNA Arrangement — jmuiuc · 2026-09-01
- NeurReps 2026 CFP: Symmetry and Geometry in Neural Representations — fatihdin4en · 2026-09-01
- Dan Luu on why software slowness is a choice, analyzing latency costs and optimization — JeremyCMorgan · 2026-09-01
- Scholar calls out LLM gibberish: reviewing papers and replies is now a waste of time — thegautamkamath · 2026-09-01
- Paper analyzes reasoning models like o1 and DeepSeek R1, probing CoT data contamination — rao2z · 2026-09-01
- Qdrant's Sept 17 stream: token-native storage claims 10-100x faster reads — qdrant_engine · 2026-09-01