Benchmark Shows TP Outperforms PP for Qwen on Dual L40S

A developer tested the multi-GPU deployment of the Qwen3.6-35B-A3B-FP8 model using two Nvidia L40S GPUs without NVLink in a Proxmox environment. Benchmarks revealed that Tensor Parallelism (TP) outperforms both Pipeline Parallelism (PP) and Data Parallelism (DP).

2026-07-29 ~ 2026-07-29 · 2 related posts

• On two L40S GPUs, vLLM tensor parallelism beat pipeline parallelism in a Qwen3.6-35B FP8 benchmark — amiitk · 2026-07-29
• Benchmarking Qwen3.6-35B on Dual L40s: Tensor Parallelism Beats Pipeline and Data Parallelism — gulensah · 2026-07-29