torch-dist-bench reveals collective communication latency and bandwidth on 8x B200

Stas Bekman shared torch-dist-bench results showing latency and bandwidth across 9 collective operations on 8x B200 GPUs, where broadcast/reduce are fastest at 12.3µs and allreduce peaks at 24.5µs.

2026-10-15 ~ 2026-10-15 · 2 related posts

• NCCL collective latency tables: all_reduce tops median latency at 24.5µs, peaks at 839 Gbps — StasBekman · 2026-10-15
• torch-dist-bench latency plots: 9 collectives benchmarked on 8x B200 — StasBekman · 2026-10-15