89% of agent tokens are repeats: prefix caching cuts input costs by ~10x, worked example
Arindam_1729 · x · 2026-10-31
Arindam1729 published a clear explainer on why prefix caching matters more for agents than chat: LLM APIs are stateless, so agents resend the whole conversation every turn — by turn 4 of a simple loop, 89% of the prompt is content the model has already read.
Growth pattern: 6,000 tokens of system prompt and tool definitions at turn 1, 1,000 tokens of history added per turn; at turn 4, 9,000 tokens sent with 8,000 already seen.
He ran the math for a 20-turn session on DeepSeek V4.1 Flash 1017 (Nebius Token Factory): 310k prompt tokens total, $0.093 in input cost with no cache vs $0.009 with every repeated prefix cached — roughly 10x savings. Caveats: caching is best-effort, unused prefixes expire within minutes, and it only works if the start of your prompts stays stable.
More from coding & agent
- Human-in-the-loop with Claude: rubber-stamping 400-line bash scripts as 'approval' — generativist · 2026-10-31
- How to actually evaluate a GraphRAG stack: multi-hop questions, conflicting facts, citations — JeremyCMorgan · 2026-10-31
- Suno ships MCP Beta, dev uses it to sync motion graphics to AI-generated music — enggirlfriend · 2026-10-31
- Count document passes, not tokens: batching questions cuts LLM scoring cost — petrusenko_max · 2026-10-31
- Open-source framework uses a second agent to catch AI slop before it hits your repo — kwerkee · 2026-10-31
- MCP-Powered Blender Demo Stuns With Realistic Metal Reflections — sidahuj · 2026-10-31