KV Cache Rules Everything: agent context re-reads cost you a lot, providers almost nothing

AccBalanced · x · 2026-11-21

@CompleteSkeptic published a deep explainer on KV cache economics, amplified by GergelyOrosz with his own speculative decoding deepdive: the biggest cost of coding agents is re-reading their own context, which costs providers near-zero to serve — explaining both why inference suddenly became highly profitable and why that won't last, why routing doesn't work, and how KV cache heavily constrains coding agent design.

Original post →

More from coding & agent

coding & agent channel →