llama.cpp ships v0.9.0, first major v1.0.0 release weeks away
ggerganov · x · 2026-11-05
llama.cpp released v0.9.0 with the first major v1.0.0 release planned within weeks.
Key highlights:
- OCP FP8 E4M3 quantization support (saturating mode, new ftype)
- New flashattnextrows op making the unified KV cache efficient for multi-sequence inference, restoring non-unified KV prefill performance
- Generic batched tensor copy API for Metal and CUDA
- Four new architectures: Clover, Limite, Metis-1.6, Mistral Large 4
- Audio expansion: Chatterbox TTS, Nemotron 3.5 Transcribe, predefined Pocket TTS voices
- Major prefill optimizations for Gemma 4 E2B/E4B
- ggml updated to v0.29.0 with FP8 support, flash attention rows, and NCCL-based multi-node tensor parallelism across CPU, CUDA, SYCL, Metal, Vulkan, OpenCL and Hexagon backends
- Server and UI fixes and refinements
Related event: llama.cpp Releases v0.9.0, v1.0.0 Coming in Weeks(2 posts)→
More from Infra
- llama.cpp v0.9.0 ships native FP8 quantization and NCCL multi-node tensor parallelism — victormustar · 2026-11-05
- Together AI ships TKCC: 1200+ TPS on DeepSeek v4.1 Flash with 2 nodes in 24 hours — simran_s_arora · 2026-11-05
- Unsloth Dynamic v3.1 GGUFs boost Qwen3.8-Flash-Next accuracy 10-15% over GSQ-RCO — danielhanchen · 2026-11-05
- Trained an MoE across two GPU clusters 25 Gbps apart — the compute islands guide — _arohan_ · 2026-11-05
- Arena data: Gemini 4 Carbon scales cheapest on hard tasks, rivals cost 3-4x more — arena · 2026-11-05
- Apple warns holiday-quarter gross margin will slide to 45% as memory chip prices surge — rohanpaul_ai · 2026-11-05