Enabling PCIe P2P on Consumer Nvidia GPUs Boosts LLM Throughput by 25%
BidonPomoev · reddit · 2026-08-09
A developer running VLLM on a server with 4x RTX 5060Ti discovered that enabling PCIe peer-to-peer (P2P) communication on consumer Nvidia cards yields significant performance gains.
Test Setup
- Hardware: AMD EPYC server (150GB/s RAM bandwidth), 4x 5060Ti 16GB in PCIe 4.0 x8
- Model: Qwen/Qwen3.6-27B-FP8 (F16 KV cache, no KV quantization)
- Parallelism: Tensor parallelism
Performance Gains
With P2P enabled, Time To First Token (TTFT) dropped significantly at 32k context depth, and prefill throughput saw a 25% boost for free.
How to Enable
- Hardware must support ReBAR (enable in BIOS).
- Install patched Nvidia community drivers.
More from Infra
- Home Assistant 2026.08 Adds Official llama.cpp Integration — ngxson · 2026-08-09
- DeepSeek Local Deployment: Troubleshooting Severe Speed Drop with Speculative Decoding — Easy_Werewolf7903 · 2026-08-09
- Fixing Black Video Outputs with MiniMax H3 on AMD GPUs — Present-Guitar-3967 · 2026-08-09
- Meituan's LongCat 2.0: Fully Trained and Inferenced on Chinese ASICs — bycloud · 2026-08-09
- Running MiniMax H3 on RTX 5090: Video-to-Video Generation Takes 20 Minutes — Chaztle · 2026-08-09
- Which 4-bit Quant is Best for MLX? Comparing Mainstream Options — True_Tangerine_4706 · 2026-08-09