MIT's QDA uses LLM agents to auto-design quantization, beating best public recipes

mit-han-lab · hf · 2026-11-19

MIT Han Lab and NVIDIA present QDA, a closed-loop system where an LLM agent designs post-training quantization recipes per model and precision. It scores candidates in minutes via KL divergence on held-out reasoning traces, checks promising ones with long generations, and builds on a library of mutually compatible PTQ methods. Across four reasoning models it improves accuracy by 0.9/1.4/2.2 points on average at W4A16/W4A8/W3A16 using 64 GPU-hours per search; on the 320B MoE GLM-5.3-Flash at NVFP4 W4A4 it recovers 98.3% of FP8 accuracy vs 96.9% for the best vendor checkpoint, with zero inference overhead. Library and harness will be open-sourced.

Original post →

More from Infra

Infra channel →