Token compression can miss the real bill: transcript resend and cache writes dominate
KitchenAmoeba4438 · reddit · 2026-07-25
Context compression often targets the wrong cost layer
The post argues that token compression usually saves the latest message, while the real bill comes from repeatedly resending the entire transcript in tool loops.
- In a long agent loop, turn 40 pays again for turns 1–39, so costs scale roughly with the square of turn count.
- Cache reads are cheaper than base input, but a high hit rate on a large prefix can still be expensive because you keep paying to reread old context.
- Cache writes are even more costly, and because Anthropic-style caching is prefix-based, editing anything inside the cached prefix can force a rewrite of the tail.
- The practical advice: order context by mutation rate, avoid editing inside a cached prefix, and summarize at a boundary where you intentionally rebuild the prefix.
The author also notes that token counters can be misleading if you are not tracking cache creation vs. cache reads, and suggests measuring cost per successful task in paired runs instead of trusting token savings alone.
More from coding & agent
- Claude Code may silently fall back from Opus 5 to Opus 4.8 on refusal — steipete · 2026-07-25
- Cloning a $13M/Year App in Under 2 Hours with AI: The Full Playbook — PrajwalTomar_ · 2026-07-25
- Stardock is using AiClairvoyance to let project-manager AIs hire AI staff — draginol · 2026-07-25
- tldraw offline tops 13,150 downloads before adding Pi agent skills — max__drake · 2026-07-25
- ComfyUI tutorial shows how to guide motion in LTX 2.3 with public nodes — nghtdrp · 2026-07-25
- Builder is making an AI receptionist for small clinics to handle calls and bookings — Icy_Current9287 · 2026-07-25