Offloading MoE experts to Threadripper Pro: a hands-on way to max out CPU cores
HankYeomans · x · 2026-10-18
A developer shares a practical trick: to see how hard Threadripper Pro cores can work, offload some MoE experts to it. This hybrid GPU+CPU expert offloading lets CPU cores fully participate in LLM inference — a useful approach for local deployments, though the post doesn't detail specific configs.
More from Infra
- Qwen3.8 27B benchmarks on NVIDIA RTX Spark: 19.3 tok/s decode, 100% accuracy — draginol · 2026-10-19
- Gemma 4 12b on NVIDIA RTX Spark laptop: 31.6 tok/s decode, 100% accuracy — draginol · 2026-10-19
- NVIDIA shows how to scale model endpoints on Kubernetes with Dynamo in 5 minutes — NVIDIAAI · 2026-10-19
- Mac Studio M5 Ultra beats dual DGX Sparks in local LLM decode speeds — SumitGup · 2026-10-19
- Generating a 5:53 music video on an AMD 7900 XTX: any tips? — wgizmo · 2026-10-19
- Qualcomm's NPU runs Qwen3 1.7B locally at 48.2 tok/s at half Nvidia's price — draginol · 2026-10-19