Gamer Builds Local LLM Home Lab with Four 16GB GPUs
HippEMechE · reddit · 2026-07-07
A Reddit user shared a local LLM home lab built on four 16GB GPUs. Using main-slot bifurcation and PCIe 3.0 x1 risers, it runs two llama.cpp instances and deploys Qwen 3.6 with speculative decoding (q40, 150k context per instance), achieving around 1000 tok/s prompt processing and 45-60 tok/s generation.
They built a backend using opencode to manage llama.cpp and token counting, estimating they have already saved about $60 in API fees.
More from Infra
- YC talk on BCI x AI says infrastructure is what really determines speed — garrytan · 2026-07-27
- A 13B model ran on a no-GPU PC by paging weights from SSD via llama.cpp — ID_R_McGregor · 2026-07-27
- llama.cpp warns that GGUFs made before a recent change must be regenerated — EconomySerious · 2026-07-27
- RTX 5090 local tests show Qwen Q6 can drop to 15 tok/s at 80k context — LFAdvice7984 · 2026-07-27
- Surprising Ubuntu Setup: NVIDIA 5090 PC Becomes the Easiest AI Rig — _xjdr · 2026-07-27
- TSMC reportedly plans 5%–10% price hikes in 2027 to cover rising costs — Beth_Kindig · 2026-07-27