Offloading Engram to DRAM boosts H200/B200 performance by up to 50%
zephyr_z9 · x · 2026-09-21
SemiAnalysis reports that offloading Engram to DRAM yields up to 50% better performance on H200, B200, B300, and GB300 NVL72. Its team, working with engineers from Pi, also upstreamed ROCm Engram DRAM offloading support to vLLM (PR 57491) with similar gains. PRs 985, 1002, 1005, and 1006 are updating default recommended recipes from NVIDIA, AMD, and SemiAnalysis to enable Engram offloading.
Related event: SemiAnalysis Tests Engram DRAM Offloading, Up to 50% Gains(2 posts)→
More from Infra
- Omarchy 4.6 lets you share local AI models across your tailnet, says DHH — alexcovo_eth · 2026-11-12
- Eric Schmidt's foundation funds French NGO fighting against data centers — StephanSturges · 2026-11-12
- Epoch: Open-Weight GLM 5.3 Hacks Six Firefox Versions for ~$1,000 Each; Claude Fable 5.2 Sets ECI Record — Epoch AI · 2026-11-12
- Polymarket prices 28% odds of a state data center moratorium; Torus raises $500M — Polymarket · 2026-11-12
- AWS-wrapper cloud startup shuts down just one month after seed round — thedealdirector · 2026-11-12
- Utah startup Torus raises $500M for shipping container-sized AI data centers deployable in 4 months — Polymarket · 2026-11-12