KV cache per token shrank 437x in 3 years — why this dev thinks AI gets cheaper
gnukeith · x · 2026-10-29
Sharing a chart of global KV cache bytes per token, gnukeith argues AI will become significantly cheaper and more capable over time rather than more expensive: DeepSeek-V1 (Nov 2023) used 389,120 bytes/token, V3.2 cut it 8.1x to 48,068, V4-Flash another 13.7x to 3,514, and V4.1-Flash (Sep 2026) just 890 bytes — roughly a 437x reduction in three years, steadily collapsing the memory and cost curve of long-context inference.
Related event: DeepSeek quantization praised as KV cache shrinks 437x in three years(2 posts)→
More from Infra
- Geekerwan Tests Show Snapdragon's 2nm 5GHz Push Failed to Deliver Efficiency Gains vs Apple and MediaTek — firstadopter · 2026-10-29
- NyoomFloat12: Lossless 12-Bit Format Speeds Up Qwen3-32B Inference Up to 2.54x — realDanFu · 2026-10-29
- Prime Inference scales from zero to 120B tokens/day on OpenRouter in one month — eliebakouch · 2026-10-29
- Durable execution meets agents: decades-old infrastructure assumptions break down — blaizedsouza · 2026-10-29
- Analyst: hyperscaler free cash flow likely inflects around 2028-29 — BenBajarin · 2026-10-29
- Unity Pitches Snowflake: Your DeepSeek Model Is 3 Months Old, We Ship OSS on Day 0 — jefrankle · 2026-10-29