llama.cpp PR refactors Intel XMX matrix engines, nearly doubles speculative decoding on B70
nickm_27 · reddit · 2026-10-23
llama.cpp PR #29864 by budiony refactors Intel XMX matrix engines into a multi-column layout, delivering a large speculative decoding speedup on Intel B70 GPUs via SYCL.
Testing with Muse Glimmer Q6K (30B) and DFlash2 across 9 task types: aggregate wall time dropped from 46.86s to 26.68s, with per-task throughput rising from 36-53 tok/s to 53-88 tok/s (summarize: 50.1 → 85.4). Token verification is now so cheap that n-max was raised, dropping the acceptance rate from 0.78 to 0.36 — yet overall throughput nearly doubled.
More from Infra
- OpenJet claims Qwen3.8-27B coding runs 1.9x faster than llama.cpp on one RTX 3090 — JeremyCMorgan · 2026-10-23
- Jensen Huang lays out NVIDIA's full Vera Rubin agentic system: NVL72, MGX, CMX and RTX — rohanpaul_ai · 2026-10-23
- ONNX to Transformers.js: running models directly in your browser — JosephJacks_ · 2026-10-23
- MiniMax M3.1 Flash free for 3 days on 0G, with 1M context and 512K output — MiniMax_AI · 2026-10-23
- "We bet on Cloudflare and it paid off": R2 and D1 at billion scale — dinasaur_404 · 2026-10-23
- Public data is drying up; private training data is turning into a browsable catalog — SucceededMind · 2026-10-23