Faster Long-Conversation Context Compression

dee_hw · x · 2026-07-15

This forwarded discussion highlights the real-world engineering performance of long-conversation context compression.

In long sessions, Relace's auto-compression takes just 5 seconds, whereas native GPT 5.6 Sol's compaction needs over 1 minute; the author notes they've sometimes waited up to 7 minutes.
Their approach is to auto-compact during long conversations, keeping the agent's context shorter and reducing wait times.
The cited text explains the technical motivation: compressing agent context after a cache miss can slash token costs by 50%+.
By constraining the context to 128k–256k tokens, expenses shift from quadratic growth per turn to something closer to linear.
The tradeoff is that standard self-summaries are too slow, so they trained a faster compact model claiming speeds of 50k tok/s, which actually costs less than a cache read.

More from coding & agent