Qwen 27B on RX 7900 XTX: Vulkan only ~4% faster than Ollama/ROCm in hands-on benchmark
AIOfficialBot · reddit · 2026-09-05
A detailed hands-on benchmark of Qwen3.8 27B (Q4KM) on Ryzen 9 9950X + RX 7900 XTX 24GB comparing Ollama/ROCm vs a from-source llama.cpp/Vulkan build (Flash Attention, q8 KV cache, all layers on GPU):
- 64K context: Ollama prompts at 215.8 t/s, generation 34.4 t/s; Vulkan 192.0 t / 35.8 t/s
- 8K context: Vulkan 230.5 t / 36.0 t/s
- Shrinking context from 64K to 8K barely moved decode speed (35.8 → 36.0 t/s)
Takeaway: plain Vulkan is only 4% faster at generation, while Ollama actually wins on 64K prompt processing; the 27B model fits entirely in VRAM at 64K context. The author wonders why others report 60–100 t/s — speculative decoding/MTP, different flags, or quants — and offers to run additional benchmarks on request.
More from Infra
- PostgreSQL 19 slips to November as LLM-assisted code review floods committers with edge cases — sanjaykalra · 2026-10-21
- Decision Models' routing classifier hits AntSeed, cutting inference cost to 24% of always-Opus — airesearch12 · 2026-10-21
- Eon cuts engineer onboarding from 2 days to 2 minutes with an MCP Gateway on Bedrock AgentCore — AWS ML Blog · 2026-10-20
- a16z's Ben Horowitz says AI buildout isn't the 90s bubble — an investor pushes back — henloitsjoyce · 2026-10-20
- Micron and Nvidia drive ~36% of S&P 500's Q3 earnings growth — Beth_Kindig · 2026-10-20
- NVIDIA lands in Europe: Jensen Huang and German chancellor discuss industrial AI at GTC Berlin — nvidia · 2026-10-20