Open-sourced Intel GPU Vulkan inference upgrades: 2x ingestion, 60%+ faster inference
Fun-Aardvark-1143 · reddit · 2026-10-16
Developer xb1n0ry open-sourced multiple upgrades to localllama's Vulkan/OpenVINO backends as a fork:
- Ingestion doubled and inference 60%+ faster on some paths
- Significantly reduced RAM cost for initial model load, plus bug fixes for broken inference paths
- Tested on 13th-gen Intel integrated graphics and B10; aims to make laptops + CPU run A303B and similar models well
- Won't merge upstream due to backlog; soliciting advice on release strategy (fork vs. his standalone inference engine) and collaborators
The author hopes to improve the open-source Intel inference ecosystem.
More from Infra
- Amazon Proposes Per-Request Validated Evidence Caching for Production RAG — _reachsumit · 2026-10-16
- LinkedIn's TransX report: sequence transduction for recsys, +6.02% CTR in A/B tests — _reachsumit · 2026-10-16
- llama.cpp PR boosts CPU inference speed with merged AVX2 tiled accumulators — jacek2023 · 2026-10-16
- Zero's next-gen IVM engine runs in a single DO, live-querying a 100GiB R2 database — threepointone · 2026-10-16
- Meme: driving to the chip factory in 2028 captures AI's compute anxiety — beffjezos · 2026-10-16
- One volunteer maintains the time zone file behind 4 billion phones — danbri · 2026-10-16