Transformers Now Calls GGML Kernels Directly, Matching llama.cpp Performance
LysandreJik · x · 2026-09-22
Hugging Face's Lysandre announced Transformers can now use GGML kernels via the kernels library at the same performance as llama.cpp — previously it only loaded GGUF files by unquantizing them. As proof of concept, only Qwen 3.5 and its MoE counterpart are supported; users can request models via GitHub issues.
More exciting: models outside llama.cpp — even without GGUF files — can leverage GGML kernels. LLM coverage in llama.cpp is already huge, but modalities with heavy pre/post-processing (raw CV, diffusion, TTS) could benefit most. Still PoC, but promising.
Related event: Transformers Natively Runs GGUF, Matching llama.cpp Performance(8 posts)→
More from Infra
- Omarchy 4.6 lets you share local AI models across your tailnet, says DHH — alexcovo_eth · 2026-11-12
- Eric Schmidt's foundation funds French NGO fighting against data centers — StephanSturges · 2026-11-12
- Epoch: Open-Weight GLM 5.3 Hacks Six Firefox Versions for ~$1,000 Each; Claude Fable 5.2 Sets ECI Record — Epoch AI · 2026-11-12
- Polymarket prices 28% odds of a state data center moratorium; Torus raises $500M — Polymarket · 2026-11-12
- AWS-wrapper cloud startup shuts down just one month after seed round — thedealdirector · 2026-11-12
- Utah startup Torus raises $500M for shipping container-sized AI data centers deployable in 4 months — Polymarket · 2026-11-12