Engram retrieval won't replace FFNs, but cutting 40-50% of HBM needs is the real win
bookwormengr · x · 2026-09-22
Responding to @teortaxesTex's point about China producing DRAM more easily than HBM, the author breaks down what Engram/n-gram retrieval actually is: not a computational system but a retrieval system over embedding meaning — adding 2-gram and 3-gram tokens to hidden states avoids many FLOPs of 'reconstructing meaning,' while attention and FFN/MoE-style concept promotion/demotion remain necessary.
Key takes:
- Engram is unlikely to fully replace FFN/MoE layers (90% of parameters); DeepSeek only reached 40% of model bits and LongCat no more than 50% of params
- The real benefit isn't parameter elimination but infrastructure savings: 40-50% less HBM storage and — even more valuable — less HBM bandwidth, the scarcer resource
- Even partial adoption meaningfully cuts memory costs for inference
Related event: DeepSeek Engram Discussion: Semantic Retrieval That Cuts HBM Needs(3 posts)→
More from Infra
- python-build-standalone enables full LTO for CPython 3.12+, modestly boosting runtime — charliermarsh · 2026-09-22
- Measured trade-offs of three REAP-pruned Qwen3.8-Flash-Next MLX builds on Apple Silicon — MensaProdigy · 2026-09-22
- Dev claims further-optimized DeepSeek V4 NVFP4 uses 190GB of 192GB VRAM — HankYeomans · 2026-09-22
- 'AWS made the industry soft': AI infra isn't mature enough to outsource the hard parts — mgill25 · 2026-09-22
- Grass network audited: 3M+ users, $32.1M revenue serving AI training data — Ronangmi · 2026-09-22
- SemiAnalysis: mapping MoE models onto inference hardware — zephyr_z9 · 2026-09-22