ExLlamaV3 MoE offloading on 9070XT doubles prefill to 2606 t/s for Qwen3.6 35B

willeyh · reddit · 2026-11-17

Reddit user willeyh reports substantial gains tuning MoE expert offloading with ExLlamaV3 on a 9070XT 16GB: running Qwen3.6 35B A3B Q4 with experts offloaded to DDR4, raising chunk size to 5120 lifted prefill from 1300 to over 2000 t/s (peaking at 2606 t/s @ 32K) while decode stayed flat. cputhreads 6 helped slightly; swizzle/gdnsub/bggdn/wide flags all regressed; cpumoepinnedarena caused core dumps. Benchmarks and tables are on GitHub (moe-offload-bench), compiled for ROCm.

Original post →

More from Infra

Infra channel →