DeepSeek V4.1 Flash inference teardown: Engram cuts 30% params, WideEP spreads 552B over 64 GPUs
teortaxesTex · x · 2026-10-21
Latent Space translated a Zhihu deep-dive on DeepSeek V4.1 Flash large-scale inference deployment, and @bookwormengr breaks down where the efficiency gains actually come from — no black magic, just simultaneous optimization on several axes:
- Engram sparse memory: cuts 25-30% of parameters (196B model) and matching compute, with embeddings selectively read and stored in DRAM rather than HBM;
- Small prefill backbone: prefill activates only 8B params/token vs 16B for decode, so just 8 GPUs on an NVL72 rack handle prefill;
- WideEP deployment: 552B expert params spread over 64 GPUs (DP 64 + EP 64), 45B per GPU mostly at 4bit — trivial against 192GB HBM per card;
- KV reuse between the causal encoder and decoder backbones further trims overhead (architecture diagram included).
More from Infra
- VectorDash Rewrites Windows Search in Rust, Adds Local Reverse Image Search and AI Chat — andy2na · 2026-10-21
- SpaceX earnings: compute buildout beats expectations, capex and debt remain the big risk — JOBhakdi · 2026-10-21
- Enterprise AI data pipelines should start with systems of record, argues Craig Mullins — craigmullins · 2026-10-21
- Musk claims SpaceX will add 200k GPUs monthly, targeting 13GW by 2027 — analysts skeptical — JOBhakdi · 2026-10-21
- SpaceX's AI business beats expectations as Q3 net income hits $4.4B — TechCrunch AI · 2026-10-21
- Together AI Berlin meetup: n8n and NVIDIA engineers on running open models in production — togethercompute · 2026-10-21