NVIDIA's Model-Optimizer unifies quantization, distillation, pruning and speculative decoding
NVIDIA · github · 2026-09-24
NVIDIA's open-source Model-Optimizer unifies SOTA compression techniques — quantization, distillation, pruning, NAS and speculative decoding — to compress models for deployment on TensorRT-LLM, TensorRT and vLLM for faster inference. The Python library has about 3.9k stars.
More from Infra
- Anthropic paid $5B for short-term compute in Q3 as spot prices run ~2x long-term deals — tengyanAI · 2026-11-11
- UK Sovereign AI invests in Fractile to build next-gen frontier AI chips — charlieharris01 · 2026-11-11
- What happened when I moved vector retrieval into Snowflake SQL instead of orchestrating RAG outside — dima806_dima · 2026-11-11
- AI chip startup Fractile raises $720M Series C for DRAM-on-processor design — Tom_Westgarth15 · 2026-11-11
- Mobile AI Factory: agentic pipeline builds on-device SLMs that beat 5-shot frontier models — kalyan_kpl · 2026-11-11
- Vivo's Starlink direct-to-cell service hits 10M users in two weeks across Brazil — XFreeze · 2026-11-11