llama.cpp Update Boosts Intel Inference Performance
pmttyji · reddit · 2026-07-15
llama.cpp recently merged a batch of updates tailored for SYCL / Intel, focusing on performance and operator support:
- Flash Attention / SDPA optimizations via the oneDNN graph API bring significant speedups on Xe2; using Qwen3.6-27B-Q80 as an example, prefill speed increased by 1.21x at p=512 and 4.26x at p=80k.
- Additional updates include: adjusting the minimum buffer size for USM system allocation, supporting OP XIELU, adding an fp16 conv2ddw kernel, and fixing precision issues with getrows across multiple quantization formats.
Overall, this is a classic local inference stack optimization, enhancing both Intel platform viability and long-context prefill performance.
More from Infra
- How to build a PostgreSQL-backed semantic search pipeline with pgvector and Ollama — KhuyenTran16 · 2026-07-21
- NeurIPS 2026 workshop calls papers on on-device intelligence — YiMaTweets · 2026-07-21
- Milled from Solid Aluminum: AI Rig Multi-GPU Case for Local Compute — dee_hw · 2026-07-21
- FutureCaribbean’s Buildathon offers $50K, H200 compute, and an NYSE pitch — HeyAmit_ · 2026-07-21
- A new series tests which data-science workflows can run on GPUs today — pandeyparul · 2026-07-21
- Former AWS operator says Bedrock margins can beat SageMaker as agentic AI lifts CPU demand — RihardJarc · 2026-07-21