llama.cpp merges dedicated Vulkan FA prefill kernel for Intel Arc — up to 15x faster
Polaris_debi5 · reddit · 2026-11-13
llama.cpp master has merged a dedicated Vulkan Flash Attention prefill kernel for Intel Arc/Xe (PR #29357), using custom shaders tuned to Intel's native matrix core shapes and SLM architecture (m16n16k16/SIMD32), and extended to single-KV-head models like MQA/MLA. A770 remains disabled due to inconsistent test results.
Key numbers:
- ARL-H Windows: qwen3 8B prefill 126.58 → 415.31 t/s (3.3x).
- B70 Pro Windows: qwen3 8B 1064.67 → 2955.83 t/s; gpt-oss 20B 2311 → 3967 t/s.
- Arc B580 (pp16384): Gemma4 E4B 116 → 1157 t/s (+897%); single-KV-head models gain most — Ling-3.0-tiny +1530% (76→1239 t/s), Gemma4 12B +1144% (52→649 t/s).
- Most mainstream 8B-class models improve 40–140%.
A major local-inference speedup for Intel Arc GPU users.
More from Infra
- llama.cpp gotcha: -ngl -1 means auto, not all — switching fixed CPU usage and 60→76 t/s — randomjapaneselearn · 2026-11-13
- MoE expert offloading on a 16GB RX 9070XT: three engines benchmarked, ExLlamaV3 leads — willeyh · 2026-11-13
- Google expands TPU v8 lineup with 8tx and 8i Max optimized chips — Beth_Kindig · 2026-11-13
- Microsoft data center outage sparks jokes about servers 'going on protest' — SuB8u · 2026-11-13
- Data centers need power – lower income, rural residents pay the price — PaulHoule · 2026-11-13
- Notion indexes 10B+ docs on turbopuffer, cuts search costs 80% — DevinLewtan · 2026-11-13