Transformers now runs GGML kernels, matching llama.cpp performance
LysandreJik · x · 2026-09-22
LysandreJik of Hugging Face announced that Transformers can now run models via GGML kernels through the kernels library, achieving the same performance as llama.cpp — instead of merely loading GGUF files and unquantizing them. As a proof of concept, two architectures are supported so far: Qwen 3.5 and its MoE counterpart.
Even more promising: models not supported by llama.cpp can still leverage its GGML kernels. Modalities like raw CV, diffusion, and TTS — which need heavy pre/post-processing and are hard to fit into llama.cpp — can reuse these high-performance kernels without GGUF files. Still proof-of-concept, but very promising. The kernels live on the HF Hub under the GGML org.
Related event: Transformers Natively Runs GGUF, Matching llama.cpp Performance(8 posts)→
More from Infra
- KubeCon keynote: computing is shifting from human speed to agent speed — HowDevelop · 2026-11-12
- Leaked memo: Meta infrastructure head Santosh Janardhan to leave in early 2027 — dinabass · 2026-11-12
- Tool-call failure rates jump 14x across providers, charts pinpoint inference stack updates — remilouf · 2026-11-12
- Omarchy 4.6 lets you share local AI models across your tailnet, says DHH — alexcovo_eth · 2026-11-12
- Eric Schmidt's foundation funds French NGO fighting against data centers — StephanSturges · 2026-11-12
- Epoch: Open-Weight GLM 5.3 Hacks Six Firefox Versions for ~$1,000 Each; Claude Fable 5.2 Sets ECI Record — Epoch AI · 2026-11-12