Decision models now run on-device in llama.cpp, says Hugging Face CEO
ivan_bezdomny · x · 2026-10-02
Hugging Face CEO Clement Delangue announced that decision models now run on-device in llama.cpp — free, fast and private — with a one-liner: llama serve -hf ggml-org/Kev-4B-GGUF.
More from Infra
- Marin's 535B MoE hero run hits ~27% MFU on 11 NVL72 racks: expert parallelism deep dive — dlwh · 2026-10-03
- The trillion-dollar AI fight shifts to 'Neoclouds', full-stack GPU infrastructure platforms — DavidLinthicum · 2026-10-02
- Strata open-source engine runs 125B Qwen3.8-Flash-Next on gaming PCs, 5k stars in 8 days — alex_verem · 2026-10-02
- AI swarms may have added 10-40% effective compute to GPU clusters in six months — 1a3orn · 2026-10-02
- Follow-up: the optimization windfall should be smaller at the frontier — 1a3orn · 2026-10-02
- Jensen Huang: Elon did in 19 days what takes others a year — DimaZeniuk · 2026-10-02