130+ benchmarks: 3×V100 homelab runs Qwen 35B at 72 tok/s, beating a $2,500 Mac Studio
eso_logic · reddit · 2026-11-17
The author benchmarked 130+ LLM configs on decommissioned enterprise GPUs (V100/P100/K80) with X99-era hardware to test if e-waste silicon can still self-host AI.
- 3×V100-16GB is the sweet spot: a full build (dual Xeon, board, PSU, cooling) costs $1,042 before RAM. Qwen 3.6 35B hits 72 tok/s (Q8, 128k context); Gemma 4 26B 75 tok/s. A $2,500 Mac Studio M5 manages only 44 tok/s on the same model.
- Stacking GPUs scales VRAM, not speed — two 16GB cards in Q4 fit 32B models but don't double generation speed.
- V100 wins on performance-per-watt over P100.
- K80s are party tricks: hitting 72B+ requires stacking many, at 1–5 tok/s.
Biggest engineering challenge is fanless cooling; the author is prototyping an active air cooler. Full graphs and power analysis on the blog.
More from Infra
- Cloudflare rolls back update after Helium browser Turnstile breakage — uwukko · 2026-11-17
- Anthropic IPO filing reportedly reveals $900.9B in purchase and lease commitments through 2045 — Polymarket · 2026-11-17
- Qualcomm demos an on-device decision model routing AI requests in 8.6ms on NPU — carrycooldude · 2026-11-17
- Power is the defining bottleneck of the AI cycle, and the grid gap is becoming a market — ditzikow · 2026-11-17
- HSBC dumps $300M of Oracle/OpenAI data center debt at steep mid-80s discount — GaryMarcus · 2026-11-17
- Fugaku supercomputer breach may have exposed data of 4,000+ users — SydSteyerhart · 2026-11-17