Cached tokens explained: agent loops overpay for the same prefix, 50x savings
Arindam_1729 · x · 2026-10-30
An article on prompt caching: every turn in an agent loop resends the system prompt, tool definitions, and full history — all of which the model already read last turn, so you pay full price repeatedly for the same prefix.
How caching works: turn 1 stores the work in the KV cache; on turn 2, matching prefix tokens are reused instead of recomputed, and only the new message is billed at full price. On Nebius Token Factory, DeepSeek V4.1 Flash 1017 bills cached input at $0.006 per 1M tokens vs $0.30 uncached — 50x cheaper.
Caveat: caching is best-effort; a repeated request can still come back with cachedtokens: 0, so log it every turn and budget at list price. No cache keys or opt-in needed.
Related event: Prefix Caching Could Cut Agent Loop Costs 50x(2 posts)→
More from coding & agent
- Lean 4 formalization of an asymmetric metric-tensor physics paper hits GitHub — SubtleGradient · 2026-10-30
- YC-backed agent-native office suite launches free in ChatGPT and Codex — ycombinator · 2026-10-30
- Supabase never had an agent strategy — its CTO explains why it won anyway — socialwithaayan · 2026-10-30
- Understand Anything hits 88.9k stars: turn any codebase into a queryable knowledge graph — adnan_hashmi · 2026-10-30
- Scopewerk: Open-Source Rust App Puts a Hanmatek Scope on Your Desktop With MCP for AI Agents — debreuil · 2026-10-30
- The "did you" agent: a prompt for automated daily data QC — nlarusstone · 2026-10-30