Huawei Rumored Scale-Up Node with 4,096 Accelerators Could Pack 384TB of HBM
zephyr_z9 · x · 2026-09-17
Back-of-envelope math on the Huawei scale-up rumor: at fp4, one copy of large model weights needs 20TB of HBM. Assuming each hypothetical XPU carries 4GB × 4Hi × 6 stacks = 96GB of HBM, a 4,096-accelerator scale-up node would total 4096 × 96 = 384TB — though the 4Hi assumption itself is disputed.
Related event: Rumor: Huawei to expand supernode to 4096 cards with 384TB HBM(2 posts)→
More from Infra
- Seagate IronWolf Pro HDDs doubled in a year, from $550 to $1,100 — HankYeomans · 2026-09-17
- Polymarket puts 31% odds on a US state enacting a data center moratorium by end of 2026 — Polymarket · 2026-09-17
- Data centers now supply roughly 45% of local tax revenue in Loudoun County, Virginia — Polymarket · 2026-09-17
- Google, Nvidia and Anthropic back Emerald AI coalition seeking 100 GW of grid capacity for data centers — TechCrunch AI · 2026-09-17
- Keeping vLLM's prefix cache warm between agent turns — bolts98 · 2026-09-17
- Running Qwen3.8-Flash-Next on 2x5080: should I move from llama.cpp to vLLM? — whatyathinkk · 2026-09-17