CASIA paper maps how long-horizon planning emerges in foundation-model agents

CASIA · hf · 2026-07-28

What the paper studies

This paper asks how multi-turn long-horizon planning emerges in foundation-model agents, and why it remains hard to improve.

Core findings

The authors build a controlled multi-turn environment to study planning across three stages: pre-training, post-training with GRPO/OPD, and multi-teacher on-policy distillation (MOPD).
During pre-training, explicit world-model construction via chain-of-thought state-transition modeling improves long-horizon generalization more than direct action prediction.
Atomic skills alone are not enough for compositional generalization, but even a small amount of long-horizon data helps a lot.
Suboptimal trajectories hurt heavily because errors compound over long horizons.
In post-training, the paper separates general planning patterns from task-specific planning knowledge using mutual information.
It identifies when post-training is unnecessary, effective, or unsupported, and finds OPD has a broader effective region than GRPO under low-quality and long-horizon settings.
Distilling unseen procedures from a teacher with different knowledge can damage the student’s prior world model without fully teaching the new skill.
MOPD can integrate capabilities by converging on shared planning patterns; compatible patterns help cross-environment generalization, partially shared patterns help continual learning, while conflicting patterns create strong interference.

More from Research