Three AMD GPUs Run a 180B MoE Locally at 19-21 t/s—But One Flag Halves It
IVequalsW · reddit · 2026-11-01
A Reddit user built a local llama.cpp server on a Broadwell Xeon with 64GB DDR4 quad-channel and three AMD GPUs (2x RX 6800 16GB + RX 580 2048SP), running Qwen Flash Next 180B Q4 GGUF via Vulkan with --moe-cache-mib 16384. It sustains 19-21 t/s generation at the full 262K context. Notable gotcha: explicitly setting context size with -c/--ctx-size tanks performance to 8 t/s—even at smaller 100K contexts—while leaving the flag unset runs fastest.
More from Infra
- Lenovo's pocket-to-cloud AI pivot: 35% of revenue now AI-related — shashib · 2026-11-01
- Beff jokes SpaceX offers the best 'time to first token' from signature to inference — beffjezos · 2026-11-01
- AI overbuild math: 2027 compute wave may need $2.5T+ revenue, Jevon's paradox debate revisited — bookwormengr · 2026-11-01
- GPU break-even token throughput up 25-83% since May, squeezing inference providers' margins — AccBalanced · 2026-11-01
- Musk Reveals Macrohard Scale: 110K GB200 and 220K GB300 GPUs — TinfoilTricorn · 2026-11-01
- RTX 4070 runs MiniMax H3: 12s video takes 1.5h, 15s takes 5h — Serenaded · 2026-11-01