456GB DeepSeek v4.1 runs locally at 40 tok/s with Threadripper + dual RTX 6000 hybrid setup
HankYeomans · x · 2026-09-22
A developer got DeepSeek v4.1 MXFP4 (about 456GB) running locally at 40 tok/s using a hybrid setup: 2x RTX 6000 Pro Max-Q for hot experts, a Threadripper Pro with 256GB CL32/6400MT RAM handling spillover experts, and Samsung 9100 NVMe for Engrams.
The goal was pushing non-GPU hardware to its limits and showing what big-memory workstation platforms can do for local LLM inference.
More from Infra
- Gas-for-AI-datacenters forum co-located with CoRL shows energy demand spillover — carlosdponx · 2026-11-12
- a16z podcast: Mike Volpi on Cisco's 70+ acquisition playbook and the AI buildout — a16z · 2026-11-12
- AMD to pass Intel in server revenue by mid-2027 as share gains 6 points a year — BenBajarin · 2026-11-12
- Study: Reflective Data Center Roofs Actually Cool Daytime Ground by 1.8-2.2°C — SumitGup · 2026-11-12
- Tsinghua's BaGuaLu-2 tames MoE expert-load spikes at runtime, accepted to SC26 — jiqizhixin · 2026-11-12
- antirez backs low-bit DeepSeek 4.1 Flash over Qwen 27B dense on Apple M5 Macs — antirez · 2026-11-12