AMD 6800H APU benchmark finds Qwen 3.6 MoE far faster than 31B Q8_0
tabletuser_blogspot · reddit · 2026-07-28
Benchmarking Gemma 4 and Qwen 3.6 MoE on an AMD 6800H APU
A Reddit user benchmarked several models on a mini-PC with an AMD Ryzen 7 6800H using llama.cpp + Vulkan on Linux/Kubuntu, relying entirely on shared system memory (UMA) and the Radeon 680M iGPU.
Main findings
- Memory allocation from 1 GB to 16 GB did not change inference speed in this setup.
- The user compared NVFP4, Q4K, and Q80 quantizations to find the best speed/quality tradeoff.
- MoE helped a lot on APU/iGPU hardware: qwen35moe 35B.A3B NVFP4 reached 15.05 t/s decoding, far ahead of gemma4 31B Q80 at 2.30 t/s.
- gemma4 26B.A4B Q40 was the fastest of the Gemma variants tested at 18.35 t/s decoding.
- Q80 was too slow for large models on this hardware, while 4-bit quantization stayed usable.
- NVFP4/MXFP4 did not outperform Q40 in these Vulkan/AMD driver tests; the user suspects further optimization is needed.
Practical takeaway for APU users
- Avoid Q8 for models above 20B parameters.
- Prefer MoE models when you want more parameter count without killing throughput.
- Stick to 4-bit quantization if you want decoding above a comfortable reading speed.
More from Infra
- fmgo: call Apple's on-device Foundation Models from Go with no CGO and no Swift — Super_Run_8466 · 2026-09-23
- Huawei unveils Peerium architecture: nested BSP unifies million processors into one computer — Dr_Singularity · 2026-09-23
- Grok explains why DeepSeek picked DualPipe + ZeRO-1 over ZeRO-3 on 2048 H800s — TheZachMueller · 2026-09-23
- AI costs fall 47% per quarter, 4x faster than DNA sequencing: Epoch AI — daveholtz · 2026-09-23
- M5 Ultra LLM test: 4x faster prompt processing, but double the power draw — DigitalguyCH · 2026-09-23
- $500 of Dell OptiPlexes become a diskless netboot lab where AI agents can't brick the hardware — colinmcnamara · 2026-09-23