DeepSeek V4.1 Flash tech report: KV cache compression lets 552B beat 1.6T
机器之心 · wechat · 2026-09-12
Machine Heart's deep dive into the DeepSeek-V4.1-Flash tech report: a 552B-backbone + 196B Engram multimodal MoE that beats the 1.6T-parameter V4 Pro (scoring 40 on ArtificialAnalysis) by compressing KV cache to the extreme — global KV down to 890 bytes/token, 437x less than V1.
Key techniques:
- CED (Causal Encoder-Decoder): splits 40 layers in half; decoder KV entries are projected from layer-20 hidden states, so prefill only runs 20 layers — near 2x complexity reduction (building on Microsoft's YoCo but with per-layer projections).
- CSA2 sparse attention: fills all three compression dimensions (entry, sequence, layer) via statically assigned Full/Reindex/Reuse per-layer modes, plus FP4-quantized mainKV (E2M1 with per-16-channel E4M3 scales).
- SWA Bounded Replay: accepts approximation, replaying only the last 128 tokens and moving SWA KV out of persistent cache (DRAM pool, minutes-level TTL), cutting persistent KV to 1/8 of V4-Flash.
Other highlights: Single-Pass mHC halves activation memory traffic; the Engram N-gram-hash memory module ships in a real model for the first time, offloading static knowledge to host memory; the DSpark speculative decoding module is trained post-pretraining and accelerates both serving and RL rollouts. Post-training openly has "no algorithmic novelty," but introduces a 1–100 reasoning effort scalar: raising effort from 25 to 100 lifts average Pass@1 across eight reasoning benchmarks from 67.1% to 76.3% at 2.5x output tokens; API tiers max/high/low map to b=100/75/50. Context scaling from 4K to 1M adds only 1/4 decode FLOPs per token; despite verbosity (89k tokens/task) it costs just $0.27 per task.
More from Infra
- Mooncake now the default KV cache solution across top 3 LLM inference engines — zhyncs42 · 2026-09-12
- MiniMax H3 video gen on rented GPUs: 4090 does a clip for $0.013, L40S matches 4090 speed — Worldly_North_7213 · 2026-09-12
- Samsung's EUV photomask supply chain chart omits downstream players like Veeco — PAstynome · 2026-09-12
- Micron to nearly double HBM capacity to 100K wafers/month by year-end, challenging Samsung and SK Hynix — zephyr_z9 · 2026-09-12
- Wafer launches comprehensive AI performance engineering repo, starting with deep dive on Transformer inference — garrytan · 2026-09-12
- MiniMax H3 Director on a 16GB RTX 5060 Ti: 23-second video in 24 minutes — thatguyjames_uk · 2026-09-12