llama.cpp Metal optimization boosts IQ3_XXS decode speed on Apple Silicon
predatar · reddit · 2026-09-01
A submitted PR optimizes the Metal backend in llama.cpp, improving decode speed for IQ3XXS models on Apple Silicon. Benchmarks show TieL Coder 35B A3B decode speed increased from 65.6 to 73.9 tok/s. The author invites community testing and teases upcoming prefill improvements.
More from Infra
- SGLang Summit sponsors grow from 11 to 36 in one week; Lip-Bu Tan, Lilian Weng to speak — BanghuaZ · 2026-10-18
- Yacine jokes his datacenter KPI is gigawatt-hours: get paged if not burning enough power — yacineMTB · 2026-10-18
- 1.68x inference speedup drops, directly accelerating GRPO post-training — yacineMTB · 2026-10-18
- OpenRouter is only a fraction of the total addressable market, poster argues — sloppenheimer · 2026-10-18
- Developer builds change alerts for LLM prices and retirements across Bedrock, Azure and Vertex — jsandhu0780 · 2026-10-18
- 12.3M tokens in 27 minutes for $0.14: agent session with 99.6% cache hit — teortaxesTex · 2026-10-18