Switching to EPYC: 4x Faster Local LLM Decode with Full Tuning Notes
mrgreatheart · reddit · 2026-11-01
With GPUs unchanged (RTX 3090, 5070 Ti, 2×5060 Ti), the author moved from Core Ultra 7/Z890 to an EPYC 7443 + Supermicro H12SSL with 256GB 8-channel DDR4 ECC. Tuning steps included fixing BMC fan thresholds via ipmitool, RAM OC 2666→3200, iommu=pt, APBDIS/Resizable BAR/Ten Bit Tag, and llama.cpp -ub 1024 to halve system-RAM relays. Results: GLM-5.3-Flash decode 9.7→41 tok/s (4.2×), DeepSeek-V4.1-Flash 9.3→35.2 (3.8×), RAM bandwidth +73%, PCIe 2×; also documented what didn't help.
More from Infra
- AI stocks shed 13% in October as Wall Street questions data center payoffs — Saul_Loveman · 2026-11-01
- Local STT shootout: Nemotron 3.5 Transcribe beats Fable on noisy audio — CryptoxPathy · 2026-11-01
- Old PC relabeled as an "AI system" sells for €5,400 after sitting at €750 — sernamenotdefined · 2026-11-01
- Turning AI data centers' waste heat into tea and saunas? A cozy proposal — emax · 2026-11-01
- Running a 22B video model on a 14 GB Vega 7 iGPU: patches, tricks, timings — shirosoro · 2026-11-01
- Intel EMIB packaging yield at 60-65% vs TSMC CoWoS's 98%, report says — rickasaurus · 2026-11-01