DeepSeek V4.1 Flash Deep Dive: KV Cache Compression at the Frontier

DeepSeek 在 V4.1 Flash 技术报告中公布了一个以 KV cache 压缩为核心设计目标的旗舰级模型，@nrehiew 随后发布系列技术长文逐层拆解，@demianai 转发的 Mhr1036 分析与 @teortaxesTex 转发的 arjunkocher 架构详解也给出互补视角。社区共识是：该模型通过架构层面的多项创新，同时压低了长上下文的计算与内存开销。

已确认

• 模型为 552B 参数 MoE 骨干，支持百万 token 上下文窗口；Mhr1036 指出长上下文带来「为保留上下文付费」的第二重成本，而该设计同时压缩计算与内存两端
• 核心架构为非对称因果编码器-解码器（CED/decoder-decoder）：40 层网络分为下半 20 层 encoder 与上半 20 层 decoder；@nrehiew 解释这是针对 prefill 昂贵问题的方案——下半层生成 KV cache，经逐层投影后共享给上半层，可理解为一种 YOCO 式 decoder-decoder 结构，将 KV cache 减半；arjunkocher 的拆解也确认了这一非对称设计，并提到 CSA2 注意力
• 全局注意力本质是升级版压缩注意力：三种 KV 复用变体（不复用、复用早层 KV 等）都先在小 KV 上跑 indexer 做选择，再与窗口化 KV 拼接后进入注意力
• 注意力 indexer 本身也是稀疏的：第一个 full mode indexer 先选出候选，后续层用这些候选打分，形成分层筛选结构
• 优化器采用 head-wise Muon（含视觉模型部分）；sparse attention 无需 warmup、从头训练，并配合 Sinkhorn 平衡
• 多模态方面：45T 图文 token 预训练、自研 Siglip 风格图像编码器、模态级 load balancing（@nrehiew 认为这让人联想到原始 Chameleon 的做法）
• 训练基建：一个模态计算时可并行 all-gather 另一模态的特征，对超长多图序列应用 CP（context parallelism）；部署稀疏 indexer 专用集群

为什么重要

@nrehiew 的主线判断是「对 KV cache 压缩的偏执」造就了这个超高效的旗舰模型：在百万 token 上下文成为标配的趋势下，KV cache 的读写与内存成本成为主要瓶颈，V4.1 Flash 展示了一条从架构（非对称 CED、KV 复用）、算法（稀疏分层 indexer、压缩注意力）到训练（Muon、Sinkhorn、负载均衡）的系统性降本路径，对长上下文与多模态前沿模型的工程化具有参考价值。

2026-09-11 ~ 2026-09-11 · 9 related posts

Full story(14 episodes)→

• Episode 1: DeepSeek V4.1 Flash Opens Limited Internal Beta with New Architecture and Native Multimodality(2026-09-08, 18 posts)
• Episode 2: DeepSeek V4.1 Flash Tested: Blazing 350 Tokens/s but Still Experimental(2026-09-08, 2 posts)
• Episode 3: DeepSeek cuts V4-Flash API prices with new peak/off-peak billing from Sept 10(2026-09-08, 6 posts)
• Episode 4: DeepSeek V4.1 Flash tested across tasks: near-frontier performance at a fraction of the cost(2026-09-09, 15 posts)
• Episode 5: DeepSeek releases V4.1-Flash: 552B MoE beats flagships at low cost(2026-09-09, 56 posts)
• Episode 6: DeepSeek V4.1 Flash Leak: 552B Asymmetric MoE Reportedly Rivals GPT-5.6(2026-09-10, 20 posts)
• Episode 7: DeepSeek V4.1 Flash Benchmark Results Leak, Sparking Community Discussion(2026-09-10, 2 posts)
• Episode 8: Bug Hunt Bench: DeepSeek V4.1 Flash Tops Price-Performance(2026-09-10, 8 posts)
• Episode 9: Inside DeepSeek V4.1 Flash: YOCO at its core and KV cache reuse(2026-09-10, 7 posts)
• Episode 10: DeepSeek V4.1 Flash Deep Dive: KV Cache Compression at the Frontier(2026-09-11, 9 posts)
• Episode 11: DeepSeek's New Model Report: 4x Smaller KV Cache, More Stable Training(2026-09-11, 3 posts)
• Episode 12: DeepSeek V4.1 Flash tops Vals open-source index at $0.30 per run(2026-09-11, 2 posts)
• Episode 13: DeepSeek V4.1 Flash scores 40 in AA index despite rising hallucinations(2026-09-11, 2 posts)
• Episode 14: Inside KV-Cache Sharing: How DeepSeek CED and GLM 5.2 Differ(2026-09-11, 4 posts)

• [source] DeepSeek-V4.1-Flash architecture dissected: asymmetric causal encoder-decoder, CSA2 attention, Engram memory — teortaxesTex · 2026-09-11
• DeepSeek V4.1 Flash Architecture: 552B MoE with Asymmetric 8B Read / 16B Decode Compute — demian_ai · 2026-09-11
• [source] DeepSeek V4.1 Flash notes: how obsessing over KV cache compression yields a hyper-efficient frontier model — nrehiew_ · 2026-09-11
• DeepSeek V4.1 Flash multimodal: 45T image-text tokens and modality-level load balancing — nrehiew_ · 2026-09-11
• [source] DeepSeek V4.1 Flash halves KV cache with YOCO-style decoder-decoder design — nrehiew_ · 2026-09-11
• DeepSeek V4.1 Flash's global attention: compressed attention with three KV-reuse variants — nrehiew_ · 2026-09-11
• DeepSeek V4.1 Flash: even the sparse indexer is itself sparse, with hierarchical candidate selection — nrehiew_ · 2026-09-11
• DeepSeek V4.1 Flash uses Muon optimizer and Sinkhorn balancing for 196B-param Engram — nrehiew_ · 2026-09-11
• DeepSeek V4.1 Flash infra: Siglip-style vision encoder and shadow indexer workers — nrehiew_ · 2026-09-11