Hugging Face Transformers now runs llama.cpp GGUF quants natively
ariG23498 · x · 2026-09-23
Hugging Face announces that transformers now natively supports GGUF models: pick a GGUF from the Hub and load it with the familiar frompretrained API to generate locally.
- Context: GGUF, developed by the llama.cpp team, powers local tools like Ollama, LM Studio, and Jan; publishers like Unsloth and bartowski supply ready-made GGUF checkpoints, with millions of downloads
- Performance: to match llama.cpp speed, the team reuses its ggml kernels via the kernels library and cuts overhead in generate, initially targeting Apple Silicon local inference
- Why it matters: bridges the llama.cpp and transformers/HF ecosystems, letting laptop-sized models run inside standard transformers workflows
The article also cites Julien Chaumond's demo of Qwen3.6 27B running in the Pi coding agent via llama.cpp on a MacBook Pro, performing close to the latest Opus.
Related event: Hugging Face Transformers Now Runs GGUF at llama.cpp Speed via GGML Kernels(8 posts)→
More from Infra
- Sail Research doubles Gemma 4 31B prefill MFU to 63% on TPU v6e — xennygrimmato_ · 2026-09-23
- Musk: Chinese AI leads on performance per compute, chip fix in 2-3 years — XFreeze · 2026-09-23
- Reusing connections cut LLM pre-call budget check p99 from 1278ms to 414ms — Tough_Stretch_4045 · 2026-09-23
- AI infrastructure may repay fine: deployed capex already at 121% revenue coverage — FinanceYF5 · 2026-09-23
- Transformers now natively supports GGUF, beating llama.cpp on one M2 Max benchmark — Disastrous-Work-1632 · 2026-09-23
- Goldman: US to Outspend China $806B to $110B on AI Infra in 2026, China Fights Back on Efficiency — FinanceYF5 · 2026-09-23