llama.cpp Update Boosts Intel Inference Performance
pmttyji · reddit · 2026-07-15
llama.cpp recently merged a batch of updates tailored for SYCL / Intel, focusing on performance and operator support:
- Flash Attention / SDPA optimizations via the oneDNN graph API bring significant speedups on Xe2; using Qwen3.6-27B-Q80 as an example, prefill speed increased by 1.21x at p=512 and 4.26x at p=80k.
- Additional updates include: adjusting the minimum buffer size for USM system allocation, supporting OP XIELU, adding an fp16 conv2ddw kernel, and fixing precision issues with getrows across multiple quantization formats.
Overall, this is a classic local inference stack optimization, enhancing both Intel platform viability and long-context prefill performance.
More from Infra
- Anthropic's S-1 Reveals Over $600B in Infrastructure Commitments Across Google, Amazon, Microsoft — cedric_chee · 2026-10-26
- Anthropic S-1 reveals 48 cents of compute cost per dollar of Q2 revenue — nmasc_ · 2026-10-26
- Persistent AI joins UR network to route private AI inference with encrypted, IP-hidden traffic — 0xSammy · 2026-10-26
- Anthropic's disclosed compute commitments: $114.5B Google, $161.2B Broadcom top the list — BenBajarin · 2026-10-26
- SpaceX strikes $40B deal with Apollo Global to lease AI chips and data center hardware — Polymarket · 2026-10-26
- SpaceX AI revenue up 830%, Claude designs 318K antivenom proteins in 64 hours — PeterDiamandis · 2026-10-26