4-GPU server topology: one Gen5 switch covers it, benchmark all-reduce before buying three
knowrohit07 · x · 2026-10-03
A hands-on discussion on PCIe topology for multi-GPU servers: the poster argues that with 4 cards, a single Gen5 switch with 4 x16 ports covers everything—cross-card traffic never touches the CPU, and the jam only appears when cards are split across switches. With 96GB cards, most serving setups run TP2 or independent replicas, so cross-card traffic is minimal. He also notes Supermicro H12 boards are Gen4 (half the bandwidth) while H13 is Gen5, matching the reported 53-56GB/s figures, and advises benchmarking all-reduce on one switch before spending money on three.
More from Infra
- Report: xAI to bring ~420,000 Nvidia GPUs online in November to meet compute commitments — mark_k · 2026-10-03
- Open-source k6-based mcpload stress-tests MCP servers: 3,780 sessions, 0 errors — AKM121001 · 2026-10-03
- AI Conference Day 2: Agent Inference Costs and Observability Steal the Show — sanjaykalra · 2026-10-03
- Apple M5 Ultra 256GB Context Benchmark Shows Strong Prefill Speed on Quantized Qwen — antirez · 2026-10-03
- Fractile Founder on Winning the AI Chip Market and the Memory Bandwidth Bottleneck — saranormous · 2026-10-03
- SoftBank pays $3.1B for data-center investment manager as buildout runs on debt — YvesMulkers · 2026-10-03