DeepSeek V4.1 Flash Architecture: 552B MoE with Asymmetric 8B Read / 16B Decode Compute
demian_ai · x · 2026-09-11
A detailed breakdown of DeepSeek V4.1 Flash's architecture, designed to cut both compute and memory costs of a million-token context window.
Key points:
- Asymmetric compute budgets: a 552B-parameter MoE backbone activates 8B parameters per input token and 16B per output token — valuable for agents that consume long documents, repos, and tool results before producing short responses.
- Encoder supplies decoder memory: the 40-layer network splits into a 20-layer causal encoder and 20-layer decoder, with the decoder's global KV cache built from the encoder's final representations.
- Attention reuse: compressed sparse attention mechanisms reduce long-context overhead.
Related event: Leaked DeepSeek V4.1 Flash Tech Report Reveals KV Cache-Obsessed Flagship(44 posts)→
More from coding & agent
- 'My intelligence stack': developer splits AI coding work across Opus 5.5, Fable and Astra with human review gates — sh_reya · 2026-10-27
- Baseten webinar: evaluate open models for coding agents by benchmarking on your own merged PRs — baseten · 2026-10-27
- Thorsten Ball on working with agents: run more evals, automate everything, use the best models — omarsar0 · 2026-10-27
- Where to draw the approval line for scheduled agent jobs — Reddit discussion — MemStrata · 2026-10-27
- DeepMind paper: LLMs can't sample randomly on their own — pass random numbers from code instead — rohanpaul_ai · 2026-10-27
- Retrieval's bottleneck is shifting from finding to selecting, kapa.ai argues — emil_sorensen · 2026-10-27