Token compression can miss the real bill: transcript resend and cache writes dominate

KitchenAmoeba4438 · reddit · 2026-07-25

Context compression often targets the wrong cost layer

The post argues that token compression usually saves the latest message, while the real bill comes from repeatedly resending the entire transcript in tool loops.

In a long agent loop, turn 40 pays again for turns 1–39, so costs scale roughly with the square of turn count.
Cache reads are cheaper than base input, but a high hit rate on a large prefix can still be expensive because you keep paying to reread old context.
Cache writes are even more costly, and because Anthropic-style caching is prefix-based, editing anything inside the cached prefix can force a rewrite of the tail.
The practical advice: order context by mutation rate, avoid editing inside a cached prefix, and summarize at a boundary where you intentionally rebuild the prefix.

The author also notes that token counters can be misleading if you are not tracking cache creation vs. cache reads, and suggests measuring cost per successful task in paired runs instead of trusting token savings alone.

Original post →

Token compression can miss the real bill: transcript resend and cache writes dominate

More from coding & agent