Qwen3.8-27B on Strix Halo: MTP gives 3.1x speedup, ROCm FP4 fork hits 29.9 tok/s

deepu105 · reddit · 2026-08-23

A rigorous local inference benchmark of Qwen3.8-27B on a Strix Halo laptop (Radeon 8060S, 128GB unified memory), using llama.cpp's own timing block, 3 reps per config, with 85W boost verified:

Headline results:

MTP (multi-token prediction) works: Q80 jumps from 7.3 to 22.4 tok/s (3.1x) at 73% draft acceptance
The julianmb/q38rocm FP4 fork reaches 29.9 tok/s (4.1x over stock baseline) at 84% acceptance, using only 14.6GB vs 29GB for Q80
ROCm vs Vulkan decode within 1%, but Vulkan prefill lags (209 vs 277 tok/s), so ROCm stays the default

Key observations:

Draft length 5 (Q80) / 6 (FP4) beats the backend default of 3, and the optimum doesn't shift with context length
MTP's payoff shrinks as context fills: 3.1x on empty window → 2.2x at 64k → only 1.15x at a full 256k (KV cache makes verification costlier; acceptance holds)
Prefill roughly quarters from empty to 256k (277→75 tok/s); the FP4 fork falls off faster and drops below stock Q80 by 128k (70 vs 119) — its edge is decode, not prompt processing
Didn't help: KV cache q80 (slightly worse at draft 5), flash attention off (-5%)
Caveat: the FP4 fork's completion lengths drift across draft settings (152-160 tokens for the same prompt)

Runs driven via the author's LlamaStash tool with reproducible flags.

Original post →

Qwen3.8-27B on Strix Halo: MTP gives 3.1x speedup, ROCm FP4 fork hits 29.9 tok/s

More from Infra