SemiAnalysis: Engram DRAM offloading delivers up to 50% better perf, upstreamed to vLLM
bookwormengr · x · 2026-09-23
SemiAnalysis published real-world numbers on DeepSeek's Engram design: offloading Engram to DRAM yields up to 50% better performance on H200, B200, B300 and even GB300 NVL72.
Key points:
- Engram is DeepSeek's approach to reducing HBM usage in favor of DRAM/LPDDR.
- SemiAnalysis worked with NVIDIA and AMD engineers to upstream ROCm Engram DRAM offloading support into vLLM (PR 57491), seeing up to 50% gains there too.
- Default recommended recipes are being updated to use Engram offloading via PRs 985, 1002, 1005 and 1006.
The poster quips that SemiAnalysis is god-tier at news reporting but slow as a forward-looking analyst, waking up to Engram late.
Related event: SemiAnalysis Tests Engram DRAM Offloading, Up to 50% Gains(2 posts)→
More from Infra
- Photoroom reclaims ~1,000 idle GPU hours a week with SkyPilot across Slurm and Kubernetes — skypilot_org · 2026-11-12
- Cloudflare Talk: Swapping Some Agent LLM Calls for Fast Models Like Jev Gives Big Speedups — michellechen · 2026-11-12
- Cloudflare Trains Its Own Clef and Clairic Models to Complete Its Agent Stack — michellechen · 2026-11-12
- AWS open-sources 23 production-ready GenAI assets, cutting build time from months to weeks — AWS ML Blog · 2026-11-12
- Sixth Street CEO: ~$200B of Chips Sitting in Warehouses as Data Centers Lag Behind — SumitGup · 2026-11-12
- AWS CEO Matt Garman: Space Data Centers Are 10-20 Years Away, Likely Not Economically Feasible — rohanpaul_ai · 2026-11-12