Running DeepSeek V4 on 16x RTX 5060 Ti via PLX switches
Primary_Exchange21 · reddit · 2026-08-20
A user shared a detailed hardware setup using 16x RTX 5060 Ti 16GB GPUs connected via two PLX88096 switches to run DeepSeek V4 Flash-0731.
- Configuration: Uses an ASRock Rack motherboard, Xeon Gold 6330 CPU, and custom kernel parameters to force 16GB BAR1 on each card.
- Performance:
- TP8 + PP2: 500k context, 140 tks.
- TP4 + PP4: 1M context, 80 tks.
- Cost: The entire setup cost 0.6x the price of a single RTX 6000 Pro.
More from Infra
- FT warns of 'Darwin moment' for AI borrowers as CoreWeave bonds sink to distress levels — firstadopter · 2026-10-16
- Phone as interface, home servers as the lab: a two-workflow local AI setup — EAccelerate_42 · 2026-10-16
- "A datacenter is a bank": CoreWeave drama revives calls for算力 supervision — arthurcolle · 2026-10-16
- DOT-MoE: converting dense models into sparse MoE via optimal transport — burkov · 2026-10-16
- Stanford CS336: A 7B model needs 14GB for weights but 84GB to train — stanfordnlp · 2026-10-16
- New NVFP4 trick: compressing block scales saves 45-48% per expert matrix, bit-exact — TheZachMueller · 2026-10-16