Hugging Face Transformers now runs llama.cpp GGUF quants natively
ariG23498 · x · 2026-09-23
Hugging Face announces that transformers now natively supports GGUF models: pick a GGUF from the Hub and load it with the familiar frompretrained API to generate locally.
- Context: GGUF, developed by the llama.cpp team, powers local tools like Ollama, LM Studio, and Jan; publishers like Unsloth and bartowski supply ready-made GGUF checkpoints, with millions of downloads
- Performance: to match llama.cpp speed, the team reuses its ggml kernels via the kernels library and cuts overhead in generate, initially targeting Apple Silicon local inference
- Why it matters: bridges the llama.cpp and transformers/HF ecosystems, letting laptop-sized models run inside standard transformers workflows
The article also cites Julien Chaumond's demo of Qwen3.6 27B running in the Pi coding agent via llama.cpp on a MacBook Pro, performing close to the latest Opus.
Related event: Transformers Natively Runs GGUF, Matching llama.cpp Performance(8 posts)→
More from Infra
- Omarchy 4.6 lets you share local AI models across your tailnet, says DHH — alexcovo_eth · 2026-11-12
- Eric Schmidt's foundation funds French NGO fighting against data centers — StephanSturges · 2026-11-12
- Epoch: Open-Weight GLM 5.3 Hacks Six Firefox Versions for ~$1,000 Each; Claude Fable 5.2 Sets ECI Record — Epoch AI · 2026-11-12
- Polymarket prices 28% odds of a state data center moratorium; Torus raises $500M — Polymarket · 2026-11-12
- AWS-wrapper cloud startup shuts down just one month after seed round — thedealdirector · 2026-11-12
- Utah startup Torus raises $500M for shipping container-sized AI data centers deployable in 4 months — Polymarket · 2026-11-12