NVIDIA's AutoCompress Beats Minitron by Up to 6.57 Points With Hardware-Guided LLM Pruning
nvidia · hf · 2026-11-09
NVIDIA introduces AutoCompress, a hardware-guided LLM compression framework that folds latency, throughput, and memory constraints directly into differentiable architecture search, guided by serving-engine cost models. Compared to the state-of-the-art Minitron pruning framework, it improves average accuracy across eight benchmarks by 6.57, 2.35, and 0.57 points on RTX 5070, RTX 5080, and H100. It also compresses Nemotron 3.5 Super from 120B to 74B parameters for single-B200 BF16 serving at 128K context while retaining 95% of its mean benchmark score.
More from Infra
- Getting Omarchy running on Snapdragon Surface Pro: custom kernel plus SSH tunnel for OpenClaw — vincent_koc · 2026-11-09
- Ex-PlayStation Boss: A $300 PS5, Not a $1,200 PS6, Would Be True Innovation — aakashgupta · 2026-11-09
- Rumor: Megaspeed's SEA datacenter shell route triggered US crackdown, B300 spot prices soar — basedjensen · 2026-11-09
- llama-modchip Runs DeepSeek's 284B Model Entirely in 64GB RAM With Zero GPUs — solyarisoftware · 2026-11-09
- China sets fiber record: 64 Tbps each way over 1,007 km of hollow-core fiber — AIFlow_ML · 2026-11-09
- NVIDIA invests in Point2, plans investments in Motif, RLWRLD and Upstage in Korea — NVIDIA Blog · 2026-11-09