Dual R9700 AI Pro hits only ~50 t/s on Qwen3-27B-FP8 with vLLM speculative decoding
YehowaH · reddit · 2026-08-24
A user deployed official Qwen3-27B-FP8 with MTP=3 speculative decoding on two AMD R9700 AI Pro GPUs (TP=2, PCIe Gen5 x8, Ryzen 5 7400, DDR5-6000) via vLLM. Real generation throughput is only 50 t/s with 30 t/s accepted, mean acceptance length 2.7-2.8, and 60% average draft acceptance rate. The setup uses the andysalerno/r9700-serving repo (unified aiter attention, ROCm 7.14, latest vLLM/FlashAttention/aiter). They asked the community whether these numbers are reasonable and where bottlenecks might lie.
More from Infra
- SK Hynix ties new US plant to looser chip equipment exports to China, sources say — pstAsiatech · 2026-10-15
- Perplexity's new embedding models: index with a 9B model, search with 0.6B — antoine_chaffin · 2026-10-15
- Unsloth ships GGUF quants for Nemotron 3.5: Bolt-74B runs in 24GB RAM — danielhanchen · 2026-10-15
- vLLM-Omni ships Day-0 serving for FLUX 3 Image Klein with BF16, FP8, NVFP4 support — vllm_project · 2026-10-15
- NVIDIA's Nemotron 3.5 Super VL lands training and inference support on Applied Compute — ypatil125 · 2026-10-15
- CoreWeave bond yield tops 16% as loans outlast customer contracts by ~2 years — tengyanAI · 2026-10-15