Building a 7-GPU 132GB VRAM local inference rig: consolidate on EPYC or split across two systems?
infectiousstupidity · reddit · 2026-11-04
A r/LocalLLaMA user with 4x RTX 3090 + 3x RTX 3060 (132GB VRAM total) is weighing two long-term setups for local inference: consolidating all 7 GPUs onto an EPYC 7302P with an SP3 board and 128–256GB ECC DDR4 ($2500-3000 for board/memory/risers alone) for more PCIe lanes and bandwidth, versus a cheaper split — 4x 3090 on his existing X570 at PCIe 4.0 x8/x8/x4/x4 and 3x 3060 in a B460 box.
His workloads are agentic coding with STRATA and vLLM: a large MoE as planner/orchestrator, Qwen 27B dense as the implementer, and small models for routing, embeddings, reranking and voice-to-text on the 3060s. He asks whether EPYC 7302P would bottleneck concurrent models and CPU-offloaded MoEs, whether extra system RAM meaningfully expands what he can run versus keeping everything GPU-resident, and how to divide cards — explicitly requesting firsthand benchmarks over theoretical PCIe numbers.
More from coding & agent
- Founder: Vibe-coded a PMF-successful product, then hired pro devs to fix the mess — sujingshen · 2026-11-04
- One prompt to audit cross-region traffic: dev cut trans-Atlantic latency by moving a service — DanielLockyer · 2026-11-04
- Agent silently completed only a quarter of a 200-module build and stalled waiting for confirmation — Promptmethus · 2026-11-04
- ChatGPT Agent mode can scan LinkedIn and Indeed and build a tailored job-application spreadsheet — nikola_mr64990 · 2026-11-04
- Blogger auto-applied to 500 jobs via ChatGPT and got 12 interview calls in 24 hours — nikola_mr64990 · 2026-11-04
- MCP servers with hundreds of endpoints: a search-spec-call pattern that keeps context small — sandeepK1125 · 2026-11-04