DeepSeek V4.1 Flash Is an Architecture Reset: KV Cache Cut to 1/4 by Deleting Its Own Ideas
teortaxesTex · x · 2026-09-16
A Zhihu breakdown by Exhalation argues DeepSeek's V4.1 Flash is an architecture reset, not just a cheaper model: despite more total and active parameters than its predecessor, working KV cache drops to one quarter and persistent cache storage to one eighth. The gains come from deleting old modules, sharing KV states, and recomputing local context.
- MTP removed: external draft models like DSpark eroded its speculative-decoding value, and the auxiliary loss no longer justified its memory cost
- Heavily Compressed Attention removed: ambiguous global-summary role, hard to pair with FP4 storage
- Dense warmup dropped in favor of a sparser training path
teortaxesTex frames this as DeepSeek pushing the quadratic component into ever rarer operations—accepting the bitter lesson even in its own science, searching for the irreducible part.
Related event: DeepSeek V4.1 Flash resets architecture, shrinking KV cache to a quarter(5 posts)→
More from Infra
- Palmer Luckey asks: can you use datacenter RAM for things prohibited by the operator's terms of service? — avaitopiper · 2026-11-02
- Disable MTP for 20-30% faster local LLM inference on limited RAM — InternalMode8159 · 2026-11-02
- Kioxia up 216% in a year but down 68% from its June 2026 peak — zephyr_z9 · 2026-11-02
- Intel plunges 29%, worst day since the 1970s, wiping $12B off US government stake — emmanuelvivier · 2026-11-02
- Researcher hoards nearly 1TB of DRAM, refuses to sell a stick to data centers — kevinnbass · 2026-11-02
- Self-Hosting LLMs at a Law Firm: Infra and Data Governance Harder Than the Model — EducationalGroup7245 · 2026-11-02