Huawei Rumored Scale-Up Node with 4,096 Accelerators Could Pack 384TB of HBM
zephyr_z9 · x · 2026-09-17
Back-of-envelope math on the Huawei scale-up rumor: at fp4, one copy of large model weights needs 20TB of HBM. Assuming each hypothetical XPU carries 4GB × 4Hi × 6 stacks = 96GB of HBM, a 4,096-accelerator scale-up node would total 4096 × 96 = 384TB — though the 4Hi assumption itself is disputed.
Related event: Rumor: Huawei to expand supernode to 4096 cards with 384TB HBM(2 posts)→
More from Infra
- To dethrone Postgres, new databases must prioritize ecosystem integrations from day one — mgill25 · 2026-11-02
- ESP32-powered 4-inch touchscreen builds a TUI dashboard for monitoring a DGX inference cluster — CATLLM · 2026-11-02
- OpenRouter crosses 10 quadrillion annualized token run rate, doubling every 10.5 weeks — deedydas · 2026-11-02
- Palmer Luckey asks: can you use datacenter RAM for things prohibited by the operator's terms of service? — avaitopiper · 2026-11-02
- Disable MTP for 20-30% faster local LLM inference on limited RAM — InternalMode8159 · 2026-11-02
- Kioxia up 216% in a year but down 68% from its June 2026 peak — zephyr_z9 · 2026-11-02