On two L40S GPUs, vLLM tensor parallelism beat pipeline parallelism in a Qwen3.6-35B FP8 benchmark
amiitk · reddit · 2026-07-29
A Reddit user benchmarked vLLM on a Proxmox VM with two Nvidia L40S cards passed through to Ubuntu and found tensor parallelism consistently beat pipeline parallelism and data parallelism for a Qwen3.6-35B FP8 deployment.
What they ran
- Model: Qwen/Qwen3.6-35B-A3B-FP8
- Hardware: 2× Nvidia L40S, no NVLink, PCIe 4 x16, no NUMA tuning
- Stack: vllm/vllm-openai:latest
- Workload: GuideLLM sweep, 512 input tokens / 128 output tokens, rate 8, 60 seconds
Results
- TP=2: 58 successful requests, 10.518s mean latency, 82.171 ms/token, 0.95 req/s
- PP=2: 44 successful requests, 13.928s mean latency, 108.815 ms/token, 0.7167 req/s
The post asks whether the benchmark is misleading or whether TP really is the better choice in this setup, and shares the exact vLLM flags used for TP, DP, and PP experiments.
Related event: Benchmark Shows TP Outperforms PP for Qwen on Dual L40S(2 posts)→
More from Infra
- Goldman sees AI hardware stocks hit by forced selling as HBM valuations may reset higher — firstadopter · 2026-07-29
- MoonshotAI open-sources FlashKDA, a high-performance Kimi Delta Attention kernel — MoonshotAI · 2026-07-29
- SK Hynix’s $476,000 bonuses are fueling a Samsung AI chip talent war — eyishazyer · 2026-07-29
- Samsung faces a possible engineer exodus as SK Hynix shares swing 16% — eyishazyer · 2026-07-29
- AI companies are hiring thousands of electricians and carpenters — pstAsiatech · 2026-07-29
- Tencent Hunyuan open-sources AngelSpec, a full-stack speculative decoding framework — 腾讯混元 · 2026-07-29