MIT and NVIDIA's QDA uses agents to auto-design LLM quantization recipes, beating top public ones by up to 2.2 points
omarsar0 · x · 2026-11-20
MIT and NVIDIA present QDA (Closed-Loop LLM Quantization Design with Agents), handing the post-training quantization recipe-design loop to an LLM agent.
Key ideas
- The best PTQ recipe changes with every model and precision, so experts normally redesign it by hand
- QDA's agent builds and tests candidate recipes over multiple rounds
Fast evaluation
- Scoring one quantized reasoning model normally takes tens of GPU-hours; QDA scores candidates in minutes via output KL divergence from the original model on held-out reasoning traces
- Promising candidates get long generations to catch repetition loops
- A shared library of compatible public PTQ methods lets the agent combine them without manual porting
Results
- Across four reasoning models, QDA beats the strongest public recipe by 0.9, 1.4 and 2.2 points at W4A16, W4A8 and W3A16, using 64 GPU-hours per search ($53 in tokens)
- On the 320B MoE GLM-5.3-Flash at NVFP4 W4A4, QDA recovers 98.3% of FP8 accuracy vs 96.9% for the best vendor checkpoint
- Checkpoints run on existing serving kernels with no inference overhead
The library, evaluator and harness will be open-sourced; the authors call it a step toward personalized, accessible RSI.
Related event: MIT and NVIDIA Unveil QDA: Agents Automate LLM Quantization Design(2 posts)→
More from Infra
- GPU shortage debate: companies with real AI demand are still massively compute starved — matt_slotnick · 2026-11-20
- Microsoft's Bing Web IQ Goes GA on Azure: Auto Orchestration Delivers Fresh Grounding for Agents — dejanseo · 2026-11-20
- exe.dev ships Rex integration, making remote VMs the default agentic dev workstation — davidcrawshaw · 2026-11-20
- Ex-Waymo exec Abe Murray: CUDA and ISAs won't stay moats, huge chip innovation incoming — IgorCarron · 2026-11-20
- Caixin: US remote-access cutoff and Nvidia CEO exit leave China's tech giants halting overseas compute deals — Gregory_C_Allen · 2026-11-20
- PostgreSQL 19 Ships WAIT FOR LSN, Solving Read-Your-Writes on Replicas — Simon Willison · 2026-11-20