DeepSeek V4.1 Flash's Engram memory layer trades FFN compute for lookup tables, SemiAnalysis data suggests
teortaxesTex · x · 2026-09-26
A widely shared breakdown (via SemiAnalysis data, unverified by DeepSeek) claims V4.1 Flash's Engram memory layer converts common phrases into lookup tables: only 0.8B/1.6B params active per layer, with 196B as lookup tables. Moving tables to host memory on B300 cuts TP4 to TP2 (1.6x cost-efficiency), boosts KV capacity 36% on 4x GB300, and beats SSD offloading on B200 (121M vs 52M tokens per dollar). Dual GB300 nodes hit 56K tok/s prefill and 253 tok/s single-user decode. The takeaway: cheapness comes from less compute and less HBM pressure — memory bandwidth matters more than capacity. Critics note cheapness doesn't prove modeling gains.
More from Infra
- Running Qwen2.5-Coder-3B on Android: Jupyter + Pydantic Evals on-device eval pipeline — alex_b3b · 2026-11-11
- 22% of Nvidia revenue now books to Singapore, but only ~1% is local buyers — Gregory_C_Allen · 2026-11-11
- Huawei's Kirin 9050 NPU claims offline 30B MoE on smartphones, +140% perf — AIFlow_ML · 2026-11-11
- Stratechery: Intel vs AMD's diverging earnings and Intel's need for an old-new path — Stratechery · 2026-11-11
- a16z Podcast: Mike Volpi on What the 1990s Internet Boom Teaches Us About the AI Buildout — a16z Podcast · 2026-11-11
- Alibaba files to list chip-design unit T-Head in Hong Kong to fund AI push — pstAsiatech · 2026-11-11