2× R9700 local rig runs Qwen3.8-27B at 111 tok/s for ~€4k, €1k under a 5090
smallDeltaBigEffect · reddit · 2026-09-07
A Reddit user shares full specs and first-day benchmarks of a local inference rig with 2× AMD Radeon AI PRO R9700 (32GB each) — total cost €4,000, over €1k less than a single RTX 5090.
- Build: Ryzen 7500F, 64GB DDR5 6400, Asus ProArt X870E, both cards at PCIe 5.0 x8; one card runs hot, planned power-limit to 210W plus undervolting
- Qwen3.8-27B (Quark AWQ MXFP4, vLLM Radiance TP2): 111.4 tok/s single-stream decode median, ITL 1% low 77.9 tok/s, TTFT 81ms, 131k context, 4,224 tok/s prefill at 2k prompts
- Native FP8 weights: 87.6 tok/s, 16k context, 8 concurrent sequences
- Qwen3.8-Flash-Next (UD-IQ4XS GGUF, R9V fork tiered expert offload): 35.4 tok/s; surprisingly, SATA SSD offload wasn't terrible
- Use cases: deep research, summarization, image generation, light coding; next up are context degradation and KV quants
Solid reference data for cost-effective dual-AMD vLLM local deployment.
More from Infra
- Meta researcher: horrifying infra, not talent, is why OSS can't catch closed frontier — A_K_Nain · 2026-10-23
- Jane Street and NVIDIA speed up Mamba-2 with custom cuTile kernels — blelbach · 2026-10-23
- LatentLoop: asymmetric compute allocation cuts looped-MoE training cost by 19% — pmttyji · 2026-10-23
- Same AI request emits 8.5x more CO2 in Texas than in Sweden — jlundberg · 2026-10-23
- SK Hynix Pulls Plug on Solidigm US IPO as Chip Stocks Slide, Shares Down ~30% This Quarter — rohanpaul_ai · 2026-10-23
- Designer mocks Nvidia: buy idle GPUs or lose access to new ones—definitely not a bubble — michalmalewicz · 2026-10-23