llama.cpp runs faster on E-cores than P-cores in a GPU-offloaded MoE test
dir3ctly · reddit · 2026-07-25
- The author benchmarks llama.cpp on a hybrid Intel CPU setup with an RTX 5090, comparing different CPU pinning strategies while offloading a large MoE model.
- On their machine, using only E-cores outperforms mixes that include P-cores:
- 0–23 cores: 19.8 tok/s
- E-cores only (12–23): 22.7 tok/s
- P-cores plus some E-cores (0–11): 15.6 tok/s
- The poster asks why performance gets worse when P-cores are involved, noting that memory bandwidth seems to be the main bottleneck and that they are testing with Docker CPU pinning and CUDA 13.
- This is a practical tuning/debugging thread about local inference throughput and heterogeneous CPU behavior.
More from Infra
- 12 KV Cache Reduction Techniques Every AI Engineer Should Understand, Explained — blaizedsouza · 2026-09-11
- The shadow GPU capacity market is formalizing, with Meta selling excess compute to outside buyers — DavidLinthicum · 2026-09-11
- Engram's random reads don't suit SSDs; CPU-memory over NVLink could serve all 72 GPUs — bookwormengr · 2026-09-11
- 80% of the DIY LLM inference hype posters have already quit — it's brutally hard systems work — abhijithneil · 2026-09-11
- Hugging Face's Ultra Scale Playbook: a free book on training LLMs on GPU clusters — mdancho84 · 2026-09-11
- Is inference latency becoming the biggest bottleneck for production AI agents? — Euphoric_Sea632 · 2026-09-11