Why NVL72 scale-up bandwidth sits mostly idle — cutting it won't hurt inference

bookwormengr · x · 2026-10-26

A technical argument on why reducing scale-up bandwidth in large domains like NVL72 likely won't degrade performance:

In DP-EP deployments, scale-up bandwidth only moves activations between DP and EP — tiny traffic relative to available bandwidth, even at large batch sizes.
Only Tensor Parallel stresses GPU-GPU links heavily, but with large scale-up domains operators prefer DP-EP over DP-TP anyway.
Most in-domain bandwidth consumption is GPU-HBM traffic (weights and KV cache reads), a different pool from NVLink; HBM saturates while GPU-GPU links stay mostly idle.
Training uses scale-up bandwidth more heavily but still doesn't saturate it.

More from Infra