llama.cpp adds llama serve: run Inkling 2 Mini GGUF locally with one command
simonguozirui · x · 2026-11-21
julienc (Georgi Gerganov, llama.cpp creator) demos llama serve -hf ggml-org/Inkling-2-Mini-GGUF:IQ4NL — pulling the IQ4NL quant of the newly released Inkling 2 Mini straight from Hugging Face and starting a local server in one line. The announcement image shows ggml introduced Inkling 2 Mini on Nov 20, alongside the new llama serve command that further simplifies local LLM serving.
More from Infra
- Nebius says Q3 compute deals closed above $25M per megawatt, up from Q2 — Beth_Kindig · 2026-11-21
- NVIDIA lays out 8 design guidelines for sparse attention, reusing selections for 2.4x faster Top-k — NVIDIAAI · 2026-11-21
- TensorFold 1.3.2 ships 96 PRs: DeepSeek-V4.1 Metal decode 6-17% faster, 874 tok/s on Blackwell — AIFlow_ML · 2026-11-21
- Ollama CEO in Fortune: enterprise demand for open models keeps growing — ycombinator · 2026-11-21
- Nvidia plunges 13.17% in a single day to $126.75 — jfischoff · 2026-11-21
- vLLM ships Day-0 support for TML model, cuts GB200 decode latency to 3.84ms — vllm_project · 2026-11-21