CliffCompaction: Cost-Efficient Compaction for Long-Horizon Coding Agents
Trang Nguyen, Eulrang Cho, Bingqing Chen, Tim Dettmers
cs.AI, cs.LG, cs.SE
2026-09-23
CliffCompaction truncates tool I/O and never rewrites. Kimi K2.6 hits 61.42% on Terminal-Bench at 16K for about half cost; KernelBench reaches 3.58× after 400 steps.
A coding-agent trajectory can run into millions of tokens. Windows are finite, so something has to go; KV-cache reads are the expensive line item in long sessions, and any in-place edit of the prompt breaks the prefix cache and forces a re-prefill. Two common fixes are messy. Sliding windows drop old turns and invalidate the cache every time the window moves. LLM summaries can "remember" across many rounds, then drift as summary-of-summary, and the model starts treating the summary as complete and stops re-reading files.
CliffCompaction is a frontend compressor meant to ship: training-free, drop-in, no auxiliary LLM, cache-friendly. It gives up full-history recall on purpose and keeps fidelity over coverage.
Context grows untouched until a token threshold, so the cache stays valid for a whole growth segment. Compaction only truncates or drops. It never rewrites. Tool results longer than 500 characters are deleted; short ones stay (grep hits, exit codes, brief script output). Tool calls collapse to a signature: name, path, essential args. Inlined file bodies go away because the file still sits in the repo and the signature is enough to call again. Thoughts truncate to 300 characters. The system prompt and the first user task stay in full, as do the most recent K turns.
At the next compaction, the previous compacted block is discarded. Only the live session since the last event is compressed. There is never a summary of a summary. Information still leaks forward: new turns were generated with the old compacted block in view, so behavior carries a residual. The paper calls this high precision, low recall: the last two windows stay verbatim; anything older is gone from the prompt.
On uncompressed Terminal-Bench traces, tool results are 56.0% of tokens and tool calls 28.0%, 84% together. Those two classes are both the bloat and the easiest to recover by calling again. The method is wired into mini-swe-agent, OpenHands, and Terminus-2, and also shipped as a scaffold-agnostic API proxy that can sit in front of closed Claude Code.
On SWE-bench Verified, moderate thresholds barely move the score. Kimi K2.6 goes 73.87% full-context to 73.27% at 32K, and 16K stays within 2 points. GLM 5.1 and GLM 5 Turbo at 16K are in the same band. 8K is a stress test: larger drops, not a cliff. On OpenHands at 32K versus the native condenser, Kimi 71.30% to 70.20% at lower cost, GLM 5.1 71.93% to 72.20%.
On Terminal-Bench 2.0 (Terminus-2), compaction can beat full context. Kimi K2.6 full-context is 59.16% at $0.40; CliffCompaction at 32K and 16K both hit 61.42% at $0.24 / $0.19. Terminus-2's own summarizer at 16K is 55.45%. GLM 5.1 rises from 49.83% full-context to 54.33% at 16K, cost $0.54 to $0.27. On Terminal-Bench 2.1 with GLM 5.3 Flash under Claude Code, matched 45K peak context: CliffCompaction 76.69%, native auto-compact 70.97%, default 200K 73.03%.
Test-time scaling becomes a budget you can actually spend. Three 16K Kimi K2.6 rollouts plus a learned selector (Soft Group Verification) reach 69.7% practical on Terminal-Bench 2.0 for $58.01, matching Opus 4.7 at 69.4% and beating GPT-5.3 Codex at 64.7% ($64.63 once) and Opus 4.6 at 62.9%. Versus one full-context Kimi run that is +10.5 points at 1.9× cost. Three uncompacted runs are +4.8 points at 3.0× cost ($91.65). Two 16K rollouts: 65.9% at $38.67. The selector uses step/tool counts and within-group agreement such as line overlap, trained on the other model's rollouts.
On KernelBench Level 3, no-compaction hits the 256K window first: 98% of L40S runs stop at a median 99 steps, 1.30× speedup. CliffCompaction (128K) reaches 2.23× at 200 steps and 3.58× at 400, accuracy 94% / 96%, share of problems above 2× from 50% to 86%. GPT-5-mini writing Triton hits 2.09× at 200 steps versus AdaExplore's 1.78× in the matched setup. At equal spend versus full context, about 2.02× versus 1.30×. Against sliding window, summarization, and summarization plus microcompaction, CliffCompaction is the only method that stays strong and cheap: 3.58×, $8.32 per problem, fewest kernel candidates.
The context tax on coding agents sits in tool I/O and cache reads. Never rewriting, never compacting a compaction, looks crude. It is more stable than summaries because the model is not handed a slightly wrong memory and told it is complete. Cache friendliness turns extra rollouts from a money pit into something that can undercut closed frontier runs. A generic compressor beating kernel-search specialists on KernelBench is a hint that long-horizon optimization often needs "keep going" more than a new memory module. The API proxy plugs into Claude Code and Codex.
8K thresholds lose points, and short tasks barely benefit. The lowest usable budget is scaffold-dependent: system prompt and tool schemas eat tokens before step one. Practical test-time scaling uses a learned selector, not a free oracle (Kimi 16K k=3 oracle 74.2%, practical 69.7%). No comparison to trained context managers or external memory/RAG; the paper says it lacked the budget. Sliding windows win slightly on short SWE-bench trajectories and cost more than full context. KernelBench action units are not identical to search baselines, which the paper flags.