DeepSeek pushes single-token KV cache under 1KB—at brutal infra cost
kalomaze · x · 2026-09-11
Researcher kalomaze notes DeepSeek continues its efficiency trend, somehow pushing single-token KV cache storage costs into the sub-kilobyte regime—which he calls the most agonizing, gut-wrenching, misery-inducing possible training and serving infrastructure regime.
Related event: DeepSeek Shrinks Per-Token KV Cache 54x in Nine Months to Sub-KB Levels(6 posts)→
More from Infra
- Two CVSS 9.9 Wasmtime flaws allow sandbox escape, one ARM64-only since 2020 — basedjensen · 2026-10-27
- AI Infra Meme: While Countries Need TW-Scale Power for Datacenters, People Argue About Lightbulbs — mgostIH · 2026-10-27
- NVIDIA reportedly stops taking RTX 5090 orders from board partners, even backorders — chemist_slime · 2026-10-27
- Legora runs 2,000 parallel agent evals on Modal Sandboxes for legal AI — blelbach · 2026-10-27
- Cloud release velocity scores: AWS 95, Azure 90, Google Cloud 82 — DavidLinthicum · 2026-10-27
- Selling a 5090 to scale local inference: one business weighs 5 hardware paths — Anarchaotic · 2026-10-27