DeepSeek Shrinks Per-Token KV Cache 54x in Nine Months to Sub-KB Levels

According to X users zephyrz9 and teortaxesTex, DeepSeek compressed its per-token KV cache by 54x over roughly nine months, reaching sub-KB levels of about 890 bytes per token, drawing wide community attention to its inference efficiency and memory optimization.

Confirmed

• zephyrz9 reported the 54x per-token KV cache compression over nine months, reflecting rapid iteration in inference efficiency and memory optimization.
• teortaxesTex cited the specific figure: about 890 bytes per token on the new model, an unusually low footprint.
• kalomaze confirmed single-token KV storage is now under 1KB, consistent with DeepSeek's efficiency-focused track record, but noted the associated training and serving infrastructure is "agonizing" in cost and engineering complexity.
• teortaxesTex argued that an architecture-level sparsification change is highly significant, boosting token efficiency toward OpenAI-style territory and away from massive parameter-activation waste.
• Investor 0xdoug called DeepSeek definitively the third frontier lab after OpenAI and Anthropic, with a different path: OpenAI and Anthropic push the intelligence frontier at acceptable cost, while DeepSeek pushes down cost at acceptable intelligence.

Unconfirmed

• teortaxesTex's claim that the new release carries "ChatGPT fingerprints" is personal interpretation and unverified.
• Whether long-context capability holds under such extreme KV compression remains unproven; teortaxesTex explicitly awaits long-context benchmark results.

Why it matters

KV cache is the dominant source of memory usage in long-context inference; a 54x reduction means longer contexts and higher concurrency on the same hardware, directly cutting inference costs. The "third frontier lab" framing underscores DeepSeek's efficiency-first differentiation — but if it comes at the cost of long-context quality or extreme infrastructure complexity, the value proposition still needs benchmark data and deployment practice to verify.

2026-09-10 ~ 2026-09-11 · 6 related posts

Full story(18 episodes)→

• Episode 1: DeepSeek V4.1 Flash Opens Limited Internal Beta with New Architecture and Native Multimodality(2026-09-08, 18 posts)
• Episode 2: DeepSeek V4.1 Flash Tested: Blazing 350 Tokens/s but Still Experimental(2026-09-08, 2 posts)
• Episode 3: DeepSeek cuts V4-Flash API prices with new peak/off-peak billing from Sept 10(2026-09-08, 6 posts)
• Episode 4: DeepSeek V4.1 Flash Nears Frontier Models at a Fraction of the Cost, Independent Tests Show(2026-09-09, 15 posts)
• Episode 5: DeepSeek Releases Open-Source V4.1 Flash, Beating Its Own Flagship at a Fraction of the Cost(2026-09-09, 57 posts)
• Episode 6: DeepSeek V4.1 Flash Leaks: 552B Asymmetric MoE Claimed to Rival GPT-5.6 Sol(2026-09-10, 21 posts)
• Episode 7: DeepSeek V4.1 Flash Benchmark Results Leak, Sparking Community Discussion(2026-09-10, 2 posts)
• Episode 8: DeepSeek Shrinks Per-Token KV Cache 54x in Nine Months to Sub-KB Levels(2026-09-10, 6 posts)
• Episode 9: Leaked DeepSeek-V4.1-Flash Touted as New Open-Source King(2026-09-10, 3 posts)
• Episode 10: Bug Hunt Bench Ranks Frontier Coding Models on 105 Real Bugs(2026-09-10, 13 posts)
• Episode 11: DeepSeek V4.1 Flash Confirmed to Center on YOCO Architecture(2026-09-10, 7 posts)
• Episode 12: Leaked DeepSeek V4.1 Flash Tech Report Reveals KV Cache-Obsessed Flagship(2026-09-11, 44 posts)
• Episode 13: DeepSeek V4.1 Flash tops Vals open-source index at $0.30 per run(2026-09-11, 2 posts)
• Episode 14: DeepSeek launches V4.1-Flash: 552B backbone, 1M-token context(2026-09-11, 5 posts)
• Episode 15: DeepSeek V4.1 Flash scores 40 in AA index despite rising hallucinations(2026-09-11, 2 posts)
• Episode 16: Inside KV-Cache Sharing: How DeepSeek CED and GLM 5.2 Differ(2026-09-11, 4 posts)
• Episode 17: DeepSeek 4.1 Flash Shows Big Gains in Kernel Engineering and Cost(2026-09-11, 2 posts)
• Episode 18: SGLang delivers Day 0 support for DeepSeek V4.1 Flash, hitting 873 tok/s(2026-09-12, 2 posts)

• [source] DeepSeek cut per-token KV cache size by 54x in nine months — zephyr_z9 · 2026-09-10
• Analyst: DeepSeek's latest change is a big win for token efficiency, moving toward OpenAI's regime — teortaxesTex · 2026-09-10
• DeepSeek's new release shows ChatGPT fingerprints, token efficiency set to jump — teortaxesTex · 2026-09-10
• [source] DeepSeek's 890 bytes per token sparks demand for long-context evals — teortaxesTex · 2026-09-10
• [source] DeepSeek pushes single-token KV cache under 1KB—at brutal infra cost — kalomaze · 2026-09-11
• DeepSeek cut KV cache per token 54x in 9 months, called the third frontier lab — max_paperclips · 2026-09-11