DeepSeek-V4.1 Flash deep dive: pushing KV cache compression to the limit at 420 tok/s
teortaxesTex · x · 2026-09-17
A detailed architecture analysis of the DeepSeek-V4.1 Flash technical report, arguing it is substantial enough to be called "DeepSeek-V5 Flash".
Context: the model runs at nearly 420 tokens/s in practice, and DeepSeek took all V4 Pro models offline after release — signaling an architecture-level generational change, not a post-training iteration.
Key optimizations:
- Prefill: borrowing from YOCO, the 40-layer model only passes 20 layers during prefill; activated params are 8B for prefill and 16B for decode;
- KV cache compression: stacking GQA-style head compression, CSA-style block compression, new CSA2 cross-layer compression, plus optimized Sparse Attention indexer computation;
- Precision: FP4 KV cache;
- Motivation: long-horizon agent workflows keep growing context, and KV cache storage pressure in HBM/SSD was becoming a scaling bottleneck.
Related event: DeepSeek V4.1 Flash architecture reset cuts KV cache to a quarter(5 posts)→
More from Infra
- Speculative decoding: small draft model proposes tokens, big model verifies in one pass — HowDevelop · 2026-09-17
- India to invest roughly $30 billion in building a domestic semiconductor industry — Polymarket · 2026-09-17
- Dev Inspired by AgentConf Keynote to Run All Agents Locally on Home Blackwell GPU Stack — TejasKumar_ · 2026-09-17
- Four Scheduling Techniques Flatten MoE Training Memory Peaks, Enabling 1M Context at 10.4x Throughput — Shrey Pandit · 2026-09-17
- GLM agent built its own inference infra in two weeks, tripling end-to-end throughput — jietang · 2026-09-17
- Huawei Unveils Ascend 960 Chips and SuperPoD Stack Aiming at Million-Card Clusters — 智东西 · 2026-09-17