DeepSeek cut per-token KV cache size by 54x in nine months
zephyr_z9 · x · 2026-09-10
DeepSeek has compressed per-token KV cache size by 54x over the past nine months, per a share on X — a striking inference-efficiency gain, since KV cache dominates long-context memory footprints.
More from Infra
- After Nvidia's Hugging Face buyout, devs call for a neutral alternative — hargup13 · 2026-09-10
- Google Cloud user hit with an $82k bill within 5 hours — Patient_Election2179 · 2026-09-10
- TRL ships 1M-token long-context training guide, trains Qwen3-8B on one 8-GPU node — QGallouedec · 2026-09-10
- Dual RTX Pro 6000 + Threadripper 9955W local LLM build — sanity check requested — No_Run8812 · 2026-09-10
- Screenshot surfaces rare admission of 72-hour KV cache limits in V4-era architecture — zephyr_z9 · 2026-09-10
- vLLM Ships Full Support for DeepSeek-V4.1-Flash's New Architecture — vllm_project · 2026-09-10