PivotOPD Teaches Multi-Turn Agents to Recover From Pivotal Mistakes, Beating 13 Baselines

Jeande_d · x · 2026-10-08

Researchers introduce PivotOPD, an on-policy distillation framework addressing error compounding in multi-turn agents.

Problem: An incorrect action shifts all subsequent states, so errors compound across turns. Preliminary experiments across three Qwen3 models found over half of failed rollouts contain an early "pivotal mistake" — usually still recoverable with a few guided turns.
Method: At each pivotal mistake, the teacher provides a gold action plus recovery actions for the next few turns. Preventive distillation uses gold actions with reverse KL; recovery distillation uses forward KL to transfer recovery behaviors the student rarely samples.
Results: Agents recover from pivotal mistakes over 3× more often than with standard OPD (<1% baseline). Across ALFWorld, WebShop, and Search-QA against 13 baselines, PivotOPD gains 6% on Qwen3-1.7B/8B and 3% on Nemotron-3.5.

More from coding & agent