LLM Compressor v0.14.0 makes GPTQ quantization up to 30x faster with new Triton kernel
vllm_project · x · 2026-09-24
The vLLM project released LLM Compressor v0.14.0, delivering the biggest GPTQ speedup since launch.
Key updates:
- A new Triton-based quantization kernel makes GPTQ roughly 15x faster end-to-end; batching layers that share a shape pushes this to 30x on some MoE workloads, and the remaining eager path got an independent 1.5-2x boost. Hessian offloading was removed.
- Expanded MSE/iMatrix observers widen the grid search space (a superset of the Fourosix-style strategy) and beat GPTQ for NVFP4 on internal benchmarks; a new Triton kernel makes MSE observation 10x faster with bitwise parity.
- REAP pruning gains distributed DDP support plus an e-Score correction.
- Support added for GLM 5.3 and Qwen3.8.
Release notes are on GitHub (vllm-project/llm-compressor, 3.8k stars).
More from Infra
- UK Sovereign AI invests in Fractile to build next-gen frontier AI chips — charlieharris01 · 2026-11-11
- What happened when I moved vector retrieval into Snowflake SQL instead of orchestrating RAG outside — dima806_dima · 2026-11-11
- AI chip startup Fractile raises $720M Series C for DRAM-on-processor design — Tom_Westgarth15 · 2026-11-11
- Mobile AI Factory: agentic pipeline builds on-device SLMs that beat 5-shot frontier models — kalyan_kpl · 2026-11-11
- Vivo's Starlink direct-to-cell service hits 10M users in two weeks across Brazil — XFreeze · 2026-11-11
- Nan Ya Plastics to spend NT$11.53B on fifth glass fabric plant on AI server demand — zephyr_z9 · 2026-11-11