GLM-4.7 quant showdown on a Radeon 680M iGPU: MXFP4 vs Q4_K_XL vs Q4_K_M
tabletuser_blogspot · reddit · 2026-08-19
Benchmarks of GLM-4.7-Flash quant variants (GGUF header reports deepseek2 30B.A3B MoE) on an Acemagic mini-PC (Ryzen 7 6800H + Radeon 680M iGPU, 64GB DDR5, Ubuntu + llama.cpp Vulkan): MXFP4 MoE (15.79GiB), UD-Q4KXL (16.31GiB), Q4KM (17.05GiB).
Results (3-run averages, Flash Attention on):
| Format | Prompt processing pp512 | Generation tg128 |
|---|---|---|
| MXFP4 MoE | 258.36 t/s | 11.66 t/s |
| Q4KM | 218.22 t/s | 12.09 t/s |
| UD-Q4KXL | 160.31 t/s | 13.13 t/s |
Analysis: models exceed iGPU memory, so performance is bound by DDR5 bandwidth (50–65GB/s); the platform lacks native FP4 (fp4:0), yet MXFP4 still wins prompt processing thanks to MoE's small active compute. Generation is purely bandwidth-bound, where Q4KXL's layout suits RADV driver prefetching best. Standard deviations are tiny across runs.
Recommendations: pick Q4KXL for chat/streaming; MXFP4MoE for RAG/long context with its 60% prompt-processing advantage.
More from Infra
- Large data centers employ only ~50-150 permanent on-site staff, Microsoft says — brezina · 2026-10-16
- Prediction: Opus 5.5-level models on laptops by 2027, backed by Qwen 27B data — cephaloform · 2026-10-16
- Oracle down 12% in a week, nearly 50% off June high as OpenAI exposure spooks market — alvelda · 2026-10-16
- US data center moratoriums top 400 — an AI bull explains why they're actually good for AI — kevinsxu · 2026-10-16
- Microsoft Research unveils Magnetar, the fastest zkVM at 10-39x speedups and post-quantum security — jedisct1 · 2026-10-16
- Bonsai 2 2B hits 120 tok/s on iPhone 18 Pro, outpaces Qwen 3.5 2B at half the memory — cephaloform · 2026-10-16