EPYC 32-core + 512GB DDR4 + 44GB VRAM: is a DIY local LLM rig still worth it?
sitytitan · reddit · 2026-11-16
A Reddit user asks for advice on running large models locally with an EPYC 7502P (32-core, 8-channel memory), 512GB DDR4, one RTX 4070 12GB and two RTX 5060 16GB cards — 44GB VRAM total. They care more about model size than tokens/sec, are confused by the many inference implementations, and wonder whether such a build is still useful now that Project Maya can stream models straight from NVMe.
More from Infra
- Inside SGLang Summit 2026: 23 sessions in one track and a striking open-source turnout — ying11231 · 2026-11-16
- Forensic Analysis: Top Chinese AI Models Fingerprinted to Export-Banned NVIDIA Blackwell GPUs — elie · 2026-11-16
- Samsung open-sources LittleBit: 13B LLM under 1GB with sub-bit weights and XOR compute — dr_cintas · 2026-11-16
- Samsung to invest $1.9B in Austin to make chips for SpaceX Starlink satellites — elonmusk · 2026-11-16
- Jensen Huang rumored plotting return to Nvidia CEO as Musk, Sweeney back public letter — The Verge AI · 2026-11-16
- Open-source ComfyUI node offloads text encoders to a spare GPU on another machine — randomlyfixed · 2026-11-16