Lambda benchmark: RoCE delivers ~70% lower throughput than InfiniBand on B200 nodes

TheZachMueller · x · 2026-11-21

TheZachMueller rented two same-region on-demand Lambda nodes and benchmarked interconnect impact, citing Lambda's official comparison: at 250 concurrent users on 2x HGX B200 nodes, a RoCE deployment delivered about 70% lower throughput and 2.5x slower time-to-first-token than NVIDIA Quantum InfiniBand. Controlled variables: same model (nvidia/GLM-5.2-NVFP4), same serving stack (SGLang TP16/DP16/EP16), same benchmark (AIPerf, 8192 in / 1024 out tokens) — the interconnect was the variable, though he notes it wasn't the only factor behind the gap.

Related event: Cloud B200 Test: RoCE Throughput Falls Far Short of InfiniBand(2 posts)→

Original post →

More from Infra

Infra channel →