Transformers now runs GGML kernels, matching llama.cpp performance
LysandreJik · x · 2026-09-22
LysandreJik of Hugging Face announced that Transformers can now run models via GGML kernels through the kernels library, achieving the same performance as llama.cpp — instead of merely loading GGUF files and unquantizing them. As a proof of concept, two architectures are supported so far: Qwen 3.5 and its MoE counterpart.
Even more promising: models not supported by llama.cpp can still leverage its GGML kernels. Modalities like raw CV, diffusion, and TTS — which need heavy pre/post-processing and are hard to fit into llama.cpp — can reuse these high-performance kernels without GGUF files. Still proof-of-concept, but very promising. The kernels live on the HF Hub under the GGML org.
More from Infra
- Unsloth GGUFs run Qwen-Image-2.1 locally on as little as 6GB VRAM — danielhanchen · 2026-09-23
- H Company trains computer-use agents on SkyPilot: thousands of sub-second sandboxes — skypilot_org · 2026-09-23
- MiniMax H3 video gen runs locally on M5 Ultra: 768p in ~2m22s with optimizations — bakawolf123 · 2026-09-22
- Cisco: Agentic AI to Drive 9X Enterprise Traffic Growth by 2035 vs 2.5X Without — Beth_Kindig · 2026-09-22
- Cloudflare ships Vary support in Cache Rules to tame HTTP's 'ugliest' header — threepointone · 2026-09-22
- Skipping failed pipeline stages during training: near-baseline loss at 1% failure rate — covenant_ai · 2026-09-22