Budget MoE workstation: 384GB DDR4 + one 32GB GPU is the sweet spot
Numerous-Ad6217 · reddit · 2026-10-22
Betting that MoE will become the standard for local LLM inference, the author built a budget workstation around an AMD Radeon AI PRO R9700 (32GB VRAM), keeping active experts in VRAM and inactive experts in cheap DDR4 ECC RAM.
Key findings:
- Plotting active vs inactive weights for popular open-weight MoE models at Q4/Q8 shows a consistent linear trend with slope 0.0565 — roughly 1GB of active VRAM weights per 18GB of inactive weights in RAM.
- This puts the sweet spot at 384GB DDR4 + 32GB VRAM: 384GB of cold weights maps to 21.7GB hot weights, leaving headroom for KV cache and runtime overhead. Going to 512GB RAM would push most models' active working set against the 32GB GPU limit.
- Conclusion: a cheap EPYC box with 384GB DDR4 and a single 32GB GPU can run a wide range of large MoE models, potentially many Flash-class models at Q8.
Caveats: actual results depend on architecture, expert routing, caching efficiency, and inference backend.
More from Infra
- 8x NVIDIA DGX Spark cluster with 200GB switch becomes SF's new flex — beffjezos · 2026-10-22
- NVIDIA Cosmos-3-Nano world models go live on Reactor's robot inference cloud — chris_j_paxton · 2026-10-22
- Abacus.AI CEO beta-tests GPU-hosted apps on SuperComputer, runs 3D flying-car game — bindureddy · 2026-10-22
- 2-bit draft in VRAM + exact residual in RAM: speculative decoding for models that don't fit your GPU — Happy_Guitar3521 · 2026-10-22
- Seroter's daily list: OpenAI models land on Google Cloud, PHP CVEs, Genkit Go 1.14 — rseroter · 2026-10-22
- inference.net tops OpenRouter third-party hosts at 14% share, serving ~10% of daily volume — Ronangmi · 2026-10-22