Transformers Now Calls GGML Kernels Directly, Matching llama.cpp Performance for GGUF

On September 22, Lysandre, a core maintainer of Hugging Face Transformers, announced a major upgrade to GGUF support: Transformers can now call GGML kernels directly at runtime via the kernels library, achieving inference performance on par with llama.cpp. Previously, while GGUF files could be loaded, they had to be dequantized first, resulting in noticeable performance losses.

Confirmed

• Transformers has supported loading GGUF files for years, but the old approach relied on "dequantization" — restoring quantized weights to full precision — which performed significantly worse than native quantized inference
• Thanks to work by engineer @-marcsun, Transformers now invokes GGML kernels directly through the kernels library, matching llama.cpp performance
• The vast collection of GGUF quantized checkpoints produced by the llama.cpp community (millions of downloads) can now be loaded and run directly via the familiar frompretrained API
• Local inference on Mac gets faster as well

Why it matters

• Users no longer need to deploy llama.cpp separately to run GGUF quantized models — the same performance is available within the single Transformers ecosystem
• Quantized model assets accumulated by the llama.cpp community are now directly plugged into Hugging Face's flagship library, lowering the barrier for local/edge inference

2026-09-22 ~ 2026-09-22 · 6 related posts

• Transformers now runs llama.cpp GGUF quants via ggml kernels, faster local inference on Mac — LysandreJik · 2026-09-22
• Transformers Now Runs GGUF at llama.cpp Speed via GGML Kernels — LysandreJik · 2026-09-22
• Transformers Now Calls GGML Kernels Directly, Matching llama.cpp Performance — LysandreJik · 2026-09-22
• GGUF models can now run directly in Hugging Face transformers with ggml Metal kernels — pcuenq · 2026-09-22

2 near-duplicate retellings: LysandreJik · LysandreJik