Stuck at 120-160 tps on local LLM: Reddit debates AVX-512 vs AVX2 for CPU pick
catbluesgames · reddit · 2026-10-31
A Redditor running local inference with flash attention (basalt) on a 13700K + 128GB DDR5 box sees only 120-160 tps and suspects CPU-side expert computation is the bottleneck, with low GPU utilization. They're torn between an Intel 270K+ (better value, no AVX-512) and an AMD 9950X (pricier, AVX-512, 16 cores), asking how much AVX-512 actually matters for AI workloads.
More from Infra
- Removing activation bias is a free trick that improves ViT, DiT and GPT training — ZimingLiu11 · 2026-10-31
- French lawmakers approve €0.055 per gigabyte tax on internet traffic — helloplanets · 2026-10-31
- Joke: why steal catalytic converters when unmanned Waymos carry $150k+ in GPUs — vincent_koc · 2026-10-31
- Rumors say Google is killing AI Overview, citing ad click damage and compute strain — Dan_Jeffries1 · 2026-10-31
- Opera proposes WebAI standard to let websites run local models in the browser — JChataigne · 2026-10-31
- Microsoft offers voluntary severance to ~60,000 employees while shielding AI teams — The Decoder · 2026-10-31