AMD 6800H APU benchmark finds Qwen 3.6 MoE far faster than 31B Q8_0
tabletuser_blogspot · reddit · 2026-07-28
Benchmarking Gemma 4 and Qwen 3.6 MoE on an AMD 6800H APU
A Reddit user benchmarked several models on a mini-PC with an AMD Ryzen 7 6800H using llama.cpp + Vulkan on Linux/Kubuntu, relying entirely on shared system memory (UMA) and the Radeon 680M iGPU.
Main findings
- Memory allocation from 1 GB to 16 GB did not change inference speed in this setup.
- The user compared NVFP4, Q4K, and Q80 quantizations to find the best speed/quality tradeoff.
- MoE helped a lot on APU/iGPU hardware: qwen35moe 35B.A3B NVFP4 reached 15.05 t/s decoding, far ahead of gemma4 31B Q80 at 2.30 t/s.
- gemma4 26B.A4B Q40 was the fastest of the Gemma variants tested at 18.35 t/s decoding.
- Q80 was too slow for large models on this hardware, while 4-bit quantization stayed usable.
- NVFP4/MXFP4 did not outperform Q40 in these Vulkan/AMD driver tests; the user suspects further optimization is needed.
Practical takeaway for APU users
- Avoid Q8 for models above 20B parameters.
- Prefer MoE models when you want more parameter count without killing throughput.
- Stick to 4-bit quantization if you want decoding above a comfortable reading speed.
More from Infra
- MoonEP is open-sourced with a memory-saving MoE backprop design — SeunghyunSEO7 · 2026-07-28
- Amazon Makes Largest-Ever Donation to Lean FRO for Mathematically Proving AI Agent Behavior — ChrSzegedy · 2026-07-28
- LangChain says open source is in its blood as it partners with NVIDIA — Hacubu · 2026-07-28
- Morgan Stanley maps the AI infrastructure stack and who gets paid at each layer — SumitGup · 2026-07-28
- Google Cloud adds spend caps and sandboxes for untrusted agent workloads in Cloud Run — steren · 2026-07-28
- A Fields Medalist joins OpenAI as $250B data-center financing and Kimi K3 land — Dapper-Tale-4021 · 2026-07-28