Local LLM rig: Epyc 7443, 256GB RAM, and 72GB VRAM across four GPUs
mrgreatheart · reddit · 2026-10-09
A local LLM builder shares the Epyc inference rig he ordered after community advice: an Epyc 7443 (24 cores, 128 PCIe lanes), H12SSL-NT-B motherboard with dual 10GbE, 256GB of 2666MHz RAM, and 72GB of VRAM across a 3090, a 5070 Ti, and two 5060 Tis.
Upgrades: doubled PCIe bandwidth per GPU (x8 for the 5060 Tis, x16 for the others), all on proper CPU lanes; memory bandwidth up from 96 to 140GB/s practical; total pool of 328GB RAM+VRAM. He expects better prompt processing and the ability to run GLM5.3-flash and larger Qwen3.8-flash-next quants, and asks how similar rigs handle large MoE models.
More from Infra
- Oxide Computer raises $445M Series D as AI demand drives enterprises to rethink owning vs renting compute — Sethwinterroth · 2026-10-09
- Datology AI launches Curation Studio, pitching data quality as the ultimate compute multiplier — schwarzjn_ · 2026-10-09
- Impactful Scheduling for GPU Clusters: Inside AI2's New Scheduler — Hugging Face Blog · 2026-10-09
- The 'Montreal Premium': Same GPUs Cost 20%+ More Per Hour in Canada Than the US — RichardsonDx · 2026-10-09
- Supabase acquires Turso and rebuilds itself around coding agents at Select 2026 — glcst · 2026-10-09
- Epoch AI: AI costs falling 47%/quarter, 54x faster than electricity's decline — Jsevillamol · 2026-10-09