Engram's random reads don't suit SSDs; CPU-memory over NVLink could serve all 72 GPUs
bookwormengr · x · 2026-09-11
An engineer offers a technical take on deploying Engram architectures:
- Fine for personal use, risky at server scale: Engram reads are highly random, which punishes SSDs.
- CPU memory over NVLink: any of the 72 GPUs running the model backbone could read a single copy with low latency.
- In-memory database angle: storing the Engram table in something like Redis could deliver single-digit-millisecond latency and very high concurrency.
The author is looking forward to performance analyses from Fireworks AI and radixark.
Related event: Engram's random reads strain SSDs, engineers debate in-memory solutions(2 posts)→
More from Infra
- SemiAnalysis: OpenAI reportedly negotiating ~1GW TPU commitment while seeking sites — dylan522p · 2026-10-28
- I tested 11 video upscaling methods for AI video: local, Topaz and cloud compared — lopas99 · 2026-10-28
- Thermo-backed CTO says AI is in its dial-up phase, touts disaggregated compute for an ultrafast 'broadband moment' — beffjezos · 2026-10-28
- DeepSeek-V4.1-Flash Q2 runs at 41 tok/s on one RTX 3090, 90% token match vs API — musicurgy · 2026-10-28
- USearch v3 RC ships after 1,000 days, its libraries now embedded in 550K-star projects — viglovikov · 2026-10-28
- SK Hynix says memory demand will outpace supply through 2030 — Chemical_Hawk_6307 · 2026-10-28