Threadripper PRO benchmark: 24 Zen 5 cores saturate 8-channel DDR5 for CPU MoE offload
mr_il · reddit · 2026-09-10
Planning a PC to run MoE models far beyond VRAM, the author rented a Threadripper PRO 9975WX (32-core Zen 5, 8-channel DDR5) on vast.ai and benchmarked FreeToken's production W4A8 dsfp4 GEMV kernel against synthetic expert banks matching DeepSeek V4.1 Flash's geometry (384 routed experts × 18.8MB ≈ 6.7GiB, 6 experts/token).
Key findings
- Cores stop mattering at 24: 32 vs 24 cores yields +2.2–2.4% burst, 0% sustained; the workload saturates memory, not cores. The $1,200 cheaper 9965WX is the rational pick.
- The FP4 kernel reaches 80% of STREAM bandwidth (178–182 GB/s vs 222 GB/s at 24 cores).
- Thermal throttling is real: after 8 minutes of sustained load, throughput drops 17% (182 → 150 GB/s) with clocks unchanged — DRAM/IMC thermal throttling. DIMM airflow may literally be worth 20% decode speed.
In tokens/s (4.5GB routed experts per token, 24 cores): 33 tok/s at 0% expert cache hit, 83 at 60%, 133 at 75%. Versus streaming misses over PCIe 5.0 x16 (50 GB/s), the CPU path is 3–3.7× faster per missed expert — which is why FreeToken's hybrid mode computes most misses on CPU.
TL;DR: buy memory channels and DIMM cooling, not cores; the 32-core upgrade is +2% at best. Methodology and raw CSVs are published; caveats include rented single machine and container environment.
More from Embodied
- Figure's Brett Adcock teases 'max AGI' humanoid, robots roam HQ autonomously — adcock_brett · 2026-09-11
- Tesla FSD spots oncoming cars through sun glare, credits photon count reconstruction — yunta_tsai · 2026-09-11
- NVIDIA says every major robotaxi program runs on its stack, market seen at $400B by 2035 — nordicinst · 2026-09-11
- Robotics startup Skild AI hits $100M ARR just 10 months after first deployment — deepakpathak · 2026-09-11
- McKinsey report calls humanoid robots a media distraction, sees just 2% of Physical AI market by 2045 — kscottz · 2026-09-11
- Simulated zebrafish and vision-equipped robot fish reveal how the body shapes brain circuits — DoctorJosh · 2026-09-11