Intel Xeon LLM Bandwidth Halved? Local MoE Inference Tuning Log
GetOutOfMyFeedNow · reddit · 2026-08-09
A developer attempted to run DeepSeek-V4-Flash-0731 (MXFP4 format) on a Xeon w7-3465, expecting a theoretical max memory bandwidth of 153GB/s, but actual performance fell significantly short.
- Hardware: 4x RDIMM DDR5-4800, 28 physical cores, 56 logical cores.
- Bottleneck: Measured bandwidth hovered around 36-40GB/s, with inference crawling at 3-4 tokens/s.
- Tuning: Assisted by GPT, tweaking --threads and --threads-batch to match physical cores yielded a 2.4x speedup in one iteration, but overall performance still hasn't reached the machine's theoretical limits.
More from Infra
- MLCommons launches MLPerf Endpoints, a benchmark that measures inference endpoints the way buyers actually purchase them — TheKanter · 2026-10-16
- Neocloud M&A decoded: CoreWeave, Nebius, Crusoe, Nscale all bet margins vanish beyond GPU scarcity — AccBalanced · 2026-10-16
- AI-Driven Timing-Aware Power Optimization Uses Learned Ranking With Baseline-Preserving Repair — Amit Sharma · 2026-10-16
- Nscale AI datacenter plagued by coolant leaks, transformer fire, 10 safety violations — saranormous · 2026-10-16
- WSL 3.0 ships with native containers and Linux 6.18 kernel, syscall overhead down ~16% — solyarisoftware · 2026-10-16
- Are 22 GB modded RTX 2080 Ti cards a viable budget local LLM option? — Dave86ch · 2026-10-16