SteerDistill: closed-loop teacher steering recovers quantized LLM agent performance
Genming Cui · hf · 2026-11-12
Researchers at Tsinghua AIR propose SteerDistill, a quantization-aware distillation method for low-bit LLM agents.
Key ideas:
- Analyzing failed rollouts shows quantized agents overlook task requirements or environmental conditions, diverging from full-precision references
- A full-precision teacher guides the student to revise failed responses, iteratively refining guidance from student execution feedback — a closed-loop teacher steering producing locally verified behavioral corrections
- The student then trains unassisted on corrected turns and the teacher's actions at other turns, enabling unassisted inference
Results: across 5 agentic benchmarks and 3 models, SteerDistill beats the strongest quantized baseline by 1.67–2.18 average points under W4A16 and stays within one point of BF16; on τ²-Bench it outperforms QeRL by 1.84 percentage points using 67% fewer training GPUs.
More from coding & agent
- NVIDIA exec: blind AI use wreck codebases, but one-shot coding from scratch is the exception — JFPuget · 2026-11-12
- Engineers Using LLMs Should Understand Tokens, Context Windows and Hallucinations — _jaydeepkarale · 2026-11-12
- Building a Remote MCP Server for Real SaaS: OAuth 2.1, 70 Task-Shaped Tools — djskrilled · 2026-11-12
- Dev shows e-ink desk display tracking 20 Codex agent threads in real time — Dimillian · 2026-11-12
- Building a verifiable local-AI software factory on a single GPU: the judge can't be the defendant — d3nnyvg3org3 · 2026-11-12
- GraphRAG for CTOs: three retrieval patterns and a four-week test to decide — JeremyCMorgan · 2026-11-12