KV cache exposes agent economics: devs pay big for context re-reads that cost providers nothing
hackgoofer · x · 2026-09-10
@CompleteSkeptic's article "(KV) Cache Rules Everything Around Me" argues the biggest agent cost is re-reading its own context (KV cache), which costs providers nearly nothing — explaining why agents are expensive, why inference is suddenly so profitable (and why it won't last), and why routing doesn't work. KV cache also heavily constrains coding-agent design. Practical advice: measure cache hit rate and tool-call counts closely; at scale, consider self-hosting to keep the cache savings.
More from coding & agent
- Open-source skill turns AI into interactive multi-page 'Explorable Explanations' with JS games — danshipper · 2026-09-10
- kafka-mcp: An MCP Server Letting LLM Agents Inspect Kafka Topics and Safely Reset Offsets — modelcontextprotocol · 2026-09-10
- AI agents are building a new infra layer that leaves existing enterprise systems behind — matt_slotnick · 2026-09-10
- If SORs don't adapt to agents, a new platform layer will be built on top of them — matt_slotnick · 2026-09-10
- Agents will shift power to new layers above existing enterprise systems — matt_slotnick · 2026-09-10
- Agent infrastructure is where spend happens — and incumbents have no path in — matt_slotnick · 2026-09-10