Running a 177B MoE Model on Dual 3090s and an Old EPYC: 38 tok/s for $800
ludos1978 · reddit · 2026-09-13
The OP shares their hardware setup and benchmarks for running Qwen3-Flash-Next locally (177B total params / 6B active MoE, IQ4XS quantization):
- Setup: EPYC 7551 (Zen1, 32c/64T) + Supermicro H11SSL-i + 128GB DDR4-2133 in an eight-channel config + 2×RTX 3090 (48GB) + a 1500W PSU
- Approach: llama.cpp, with expert weights kept in system RAM and hot experts cached in VRAM
- Performance: 38 tok/s on a single request; two parallel requests each drop to 4 tok/s
With $800 left in the budget, the OP is torn between adding a third 3090 or upgrading to a Zen2 EPYC (drop-in on the same socket), and asks whether more/faster RAM would actually help—the goal is keeping speeds up during parallel multi-agent workloads.
More from Infra
- TSMC reportedly raising CoWoS packaging prices up to 20% next year on AI chip demand — Beth_Kindig · 2026-10-31
- effect-harness on Cloudflare: run agent loops, tools and credentials on separate VMs — samgoodwin89 · 2026-10-31
- Google's Project Suncatcher boots TPU in orbit and beams back first AI response — sundarpichai · 2026-10-31
- Together AI launches Post-Training platform for agentic RL with tool-use environments — togethercompute · 2026-10-31
- General Compute raises $100M for inference capacity as CoreWeave stock hits 12-month low — YvesMulkers · 2026-10-31
- Analyst warns of unstable AI compute feedback loop as Kioxia's NAND price warning triggers flash crash — bookwormengr · 2026-10-31