Local LLM benchmarks: Radeon R9700 across 4 engines, 7 models — serving stack beats GPU
paq85 · reddit · 2026-11-20
A detailed local LLM benchmark run of 7 models across 4 inference engines on a single Radeon AI PRO R9700.
- Headline: Paiton vLLM (3-bit weights, 4-bit KV, speculator) hits 173.8 tok/s median single-stream and 684 tok/s aggregate at 8 streams on Qwen3.8, vs 83.8 tok/s for Strata on the same model family — a 2.07x gap on identical hardware.
- DeepSeek-V4.1-Flash at full quality: 26-27 tok/s; reference RTX 5090 + llama.cpp + DFlash2: 289.2 tok/s mean decode.
- Efficiency: 0.58-0.61 tok/W single-stream, rising to 2.3-2.4 tok/W at 8 streams — concurrency is where tokens-per-watt live.
- Surprise: hidden tests pass at 84-94%, but only 0-3 of 10 tasks come out fully green.
Author's conclusion: the serving stack may matter more than the GPU.
More from Infra
- Everpure revenue jumps 51% to $1.46B as it cuts prices despite decade-high flash costs — BenBajarin · 2026-11-20
- Report: Jensen Huang weighs founding a new semiconductor company that could raise billions day one — cuijiaxun · 2026-11-20
- AMD denies short-seller report that sent stock down 13%, calls claims misrepresentations — BenBajarin · 2026-11-20
- RTX Pro 6000 listed at $8000 on eBay, buyer-beware warning — QuanstScientist · 2026-11-20
- First rendered videos from Huawei Ascend GPU surface online — GabGarrett · 2026-11-20
- Baseten's WavePP: up to 2.9x prefill throughput on same GPUs, same kernels — baseten · 2026-11-20