Engram retrieval won't replace FFNs, but cutting 40-50% of HBM needs is the real win
bookwormengr · x · 2026-09-22
Responding to @teortaxesTex's point about China producing DRAM more easily than HBM, the author breaks down what Engram/n-gram retrieval actually is: not a computational system but a retrieval system over embedding meaning — adding 2-gram and 3-gram tokens to hidden states avoids many FLOPs of 'reconstructing meaning,' while attention and FFN/MoE-style concept promotion/demotion remain necessary.
Key takes:
- Engram is unlikely to fully replace FFN/MoE layers (90% of parameters); DeepSeek only reached 40% of model bits and LongCat no more than 50% of params
- The real benefit isn't parameter elimination but infrastructure savings: 40-50% less HBM storage and — even more valuable — less HBM bandwidth, the scarcer resource
- Even partial adoption meaningfully cuts memory costs for inference
Related event: DeepSeek Engram Is Semantic Retrieval, Can Cut HBM Needs 40-50%(4 posts)→
More from Infra
- KubeCon keynote: computing is shifting from human speed to agent speed — HowDevelop · 2026-11-12
- Leaked memo: Meta infrastructure head Santosh Janardhan to leave in early 2027 — dinabass · 2026-11-12
- Tool-call failure rates jump 14x across providers, charts pinpoint inference stack updates — remilouf · 2026-11-12
- Omarchy 4.6 lets you share local AI models across your tailnet, says DHH — alexcovo_eth · 2026-11-12
- Eric Schmidt's foundation funds French NGO fighting against data centers — StephanSturges · 2026-11-12
- Epoch: Open-Weight GLM 5.3 Hacks Six Firefox Versions for ~$1,000 Each; Claude Fable 5.2 Sets ECI Record — Epoch AI · 2026-11-12