llama.cpp v0.7.0: one-liner serves Qwen3.8-Flash with MTP spec decoding on 128GB Macs
ggerganov · x · 2026-10-17
ggerganov released llama.cpp v0.7.0, showing a single command to serve ggml-org/Qwen3.8-Flash-Next-GGUF (IQ4NL) on a 128GB Mac/Spark: llama serve -hf ... --spec-type draft-mtp --lazy-mode on. Highlights the new draft-MTP speculative decoding and lazy loading running out of the box for local inference.
Related event: llama.cpp v0.7.0 Boosts MoE Decoding Speed(2 posts)→
More from Infra
- Modal explains ImageFS, the lazy-loaded filesystem behind sub-second cold starts — josh_wills · 2026-10-17
- Blackwell Ultra GPU rental prices climb 17 straight days to an all-time high — McDonaghMatthew · 2026-10-17
- Stop posting tok/s screenshots: 5 metrics a credible local AI benchmark needs — EAccelerate_42 · 2026-10-17
- 3 models, 4 engines, 4 Macs: Draw Things hits 1.8x mflux speed on Apple Silicon — liuliu · 2026-10-17
- SurrealDB's new engines hit 28.3x RocksDB point reads, Agent Memory goes live — mattturck · 2026-10-17
- Zuckerberg & Wang on Muse Wonder: Meta's $100B AI Bet Past 5GW — RylanSchaeffer · 2026-10-17