MIT's QDA uses LLM agents to auto-design quantization, beating best public recipes
mit-han-lab · hf · 2026-11-19
MIT Han Lab and NVIDIA present QDA, a closed-loop system where an LLM agent designs post-training quantization recipes per model and precision. It scores candidates in minutes via KL divergence on held-out reasoning traces, checks promising ones with long generations, and builds on a library of mutually compatible PTQ methods. Across four reasoning models it improves accuracy by 0.9/1.4/2.2 points on average at W4A16/W4A8/W3A16 using 64 GPU-hours per search; on the 320B MoE GLM-5.3-Flash at NVFP4 W4A4 it recovers 98.3% of FP8 accuracy vs 96.9% for the best vendor checkpoint, with zero inference overhead. Library and harness will be open-sourced.
More from Infra
- KAIST Proposes PROLA: Low-Rank Adaptation for Spatiotemporal Weather Downscaling — MLVLab · 2026-11-19
- WSJ: Wall Street pledged $500B for Nvidia but hasn't put up a dime — GaryMarcus · 2026-11-19
- Bypassing PyTorch with HIP kernels makes MiniMax H3 up to 3x faster on AMD R9700 — Apprehensive_Sky892 · 2026-11-19
- Ex-DeepSeek core member open-sources Linger, a local AI agent claiming 5x faster inference — Aiden_Tech_Ai · 2026-11-19
- China opens first overseas-targeted AI cluster on Huawei Ascend chips — pstAsiatech · 2026-11-19
- Tencent says US chip curbs 'unfortunate', may slow AI training short-term — pstAsiatech · 2026-11-19