Agent Context Compaction Halves Costs
appenz · x · 2026-07-14
This post discusses agent 上下文压缩（compact）: when an agent experiences a cache miss due to prolonged inactivity, performing a compaction pass first can reduce token costs by 50% 以上.
Key points:
- Simulations using real Fable 5 traces showed that maintaining context within 128k–256k tokens shifts overall spending from 二次增长 to nearly 线性增长.
- The catch: self-summarization is slow, taking 1–2 分钟 on long traces.
- To fix this, the team trained an ultra-fast compaction model, Relace Compact, achieving 50k tok/s for token-level classification at a cost lower than cache reads.
- Pricing is about $0.20 / million input and $0.20 / million output tokens.
- It has already been integrated into Jacq, with documentation provided for integration into other products.
This is highly valuable reference material for those working on agent workflows, context management, and inference cost optimization.
Related event: Relace Compact Model Compresses Agent Context in 5 Seconds to Cut Costs(3 posts)→
More from coding & agent
- 'My intelligence stack': developer splits AI coding work across Opus 5.5, Fable and Astra with human review gates — sh_reya · 2026-10-27
- Baseten webinar: evaluate open models for coding agents by benchmarking on your own merged PRs — baseten · 2026-10-27
- Thorsten Ball on working with agents: run more evals, automate everything, use the best models — omarsar0 · 2026-10-27
- Where to draw the approval line for scheduled agent jobs — Reddit discussion — MemStrata · 2026-10-27
- DeepMind paper: LLMs can't sample randomly on their own — pass random numbers from code instead — rohanpaul_ai · 2026-10-27
- Retrieval's bottleneck is shifting from finding to selecting, kapa.ai argues — emil_sorensen · 2026-10-27