Benchmarking Qwen3.8-27B on 4x RTX 3090: Topology Matters

Mr_Moonsilver · reddit · 2026-08-18

In-depth benchmarks of Qwen3.8-27B on a 4x RTX 3090 (24GB) rig with NVLink topology reveal critical performance characteristics:

Topology > Core Count: At high concurrency, a 2-GPU TP=2 setup (using NVLink) outperforms 4-GPU TP=4 by 33–41%. The cross-PCIe all-reduce tax scales with batch size. Running 2x TP=2 instances yields 1,175 tok/s aggregate vs 418 for a single TP=4.
MTP Speculative Decoding Slows Things Down: Despite a 64% acceptance rate, MTP steps cost 3.5x more than baseline steps on Ampere + AWQ, resulting in a net loss. Users on 3090s/A-series should turn it off.
Prefill is Serial: Increasing concurrency does not improve prompt processing throughput; requests are queued single-file during prefill.
Context Capacity: With TP=4 and fp8 KV cache, the setup holds 1.45M tokens in the pool, supporting 5 concurrent 262k-context requests comfortably.

More from Infra