Transformers Natively Runs GGUF, Matching llama.cpp Performance
On September 22, Hugging Face Transformers core maintainer Lysandre announced a major GGUF upgrade: thanks to work by engineer @marcsun, Transformers can now invoke GGML kernels directly at runtime via the kernels library, achieving inference performance on par with llama.cpp. llama.cpp author Georgi Gerganov also announced the news, speeding up local inference on Mac.
Confirmed
- Transformers has long supported loading GGUF files, but the old approach dequantized the weights, with clearly worse performance than native quantized inference
- It now calls GGML kernels directly, including ggml's Metal kernels, matching llama.cpp performance
- The millions of GGUF quantized checkpoints produced by the llama.cpp community can be loaded and run via the familiar frompretrained API, generating on local machines
- Once loaded, regular Transformers APIs work as usual, easing debugging, evaluation and custom generation
- Per Hugging Face's Aritra, a benchmark showed Qwen3.5-27B outperforming llama.cpp on an M2 Max
Why it matters
- Users no longer need to deploy llama.cpp separately for quantized GGUF models; the single Transformers ecosystem delivers equal or better performance
- The quantized model assets accumulated by the llama.cpp community plug directly into Hugging Face's flagship library, lowering the barrier to local/edge inference
- Smaller quantized models plus a familiar API further reduce local deployment friction
2026-09-22 ~ 2026-09-23 · 8 related posts
Primary sources
- Transformers now runs GGUF at llama.cpp speed via GGML kernels — LysandreJik ·
- Hugging Face Transformers now runs llama.cpp GGUF quants natively — ariG23498 ·
- Transformers now natively supports GGUF, beating llama.cpp on one M2 Max benchmark — Disastrous-Work-1632 ·
- Transformers now runs llama.cpp GGUF quants via ggml kernels, faster local inference on Mac — LysandreJik · 2026-09-22
- Transformers Now Runs GGUF at llama.cpp Speed via GGML Kernels — LysandreJik · 2026-09-22
- Transformers Now Calls GGML Kernels Directly, Matching llama.cpp Performance — LysandreJik · 2026-09-22
- GGUF models can now run directly in Hugging Face transformers with ggml Metal kernels — pcuenq · 2026-09-22
- [source] Transformers now natively supports GGUF, beating llama.cpp on one M2 Max benchmark — Disastrous-Work-1632 · 2026-09-23
3 near-duplicate retellings: LysandreJik · LysandreJik · ariG23498