Transformers now runs GGUF at llama.cpp speed via GGML kernels
LysandreJik · x · 2026-09-22
Hugging Face Transformers has supported loading GGUF files for a few years, but only by unquantizing them, which hurt performance. Thanks to work by @marcsun, Transformers can now run GGUF models through GGML kernels via the kernels library, matching llama.cpp performance. Lysandre credits the @ggmlorg team for building these kernels.
Related event: Transformers Natively Runs GGUF, Matching llama.cpp Performance(8 posts)→
More from coding & agent
- User automates entire video clipping workflow with Nous Research's Hermes agent — Teknium · 2026-11-12
- As models improve, writing good evals gets harder and more important — amankhan · 2026-11-12
- Free AI game jam in SF Nov 14–15 with teams from Anthropic, Google, Meshy, OpenAI — chrisfirst · 2026-11-12
- Omarchy 4.6 lets you share local AI models across your tailnet, says DHH — alexcovo_eth · 2026-11-12
- Coding harnesses hand shell access to inference providers — 4-step gateway mitigation — shensi · 2026-11-12
- Slack's David Hoang: agents beat people on coordination, proposes 'boundary agents' — davidhoang · 2026-11-12