EMERGE-Policy: A Robot Mind Emerges Beyond a Single Policy
Zhirui Fang, Qingchi Yu, Ziyang Chen, Longfei Li, Haoran Ma, Keru Zhou, Xinrun Xu, Samith Va, Yuxuan Hu, Peixuan Song, Qiang Du, Bin Qian, Yongkang Deng, Xin Li, Yezhen Wang, Zhe Li, Hao Luo, Shuyan Li, Ziwei Wang, Weijian Deng, Xiu Li
cs.RO
2026-08-31
EMERGE-Policy treats VLAs and world models as callable skills. No extra fine-tuning: 93.9% on LIBERO-Plus versus Cosmos Policy at 82.2%, 94% on real cup stacking.
A single VLA or world-action model packs seeing, planning, moving, and checking into one forward pass. One bad step poisons the rest. Harness-style stacks put a language-model planner on top of those policies, then dump raw perception, monitoring, and verification traces back into the same context window. The planner drowns in low-level evidence and gets worse at the only job it has: pick the next capability and decide whether to recover.
The design problem is organizational. Specialized modules should work in isolation, and the planner should see only what the next decision needs. EMERGE-Policy, led by Tsinghua with Nanjing University of Science and Technology, Xi'an Jiaotong, Xidian, Harbin Institute of Technology, Peking University, and NTU, writes that organization as a graph-structured agentic framework.
A Main Agent, closed-source Codex (the paper names it codex-5.6-sol), keeps a compact active context: the instruction, current subgoal, plan status, the top of a recovery stack, distilled memory, recent structured evidence, and one-line skill signatures. Instructions become subgoals with an explicit completion check. Each subgoal is a triple of target, intended change, and criterion, stored in PLAN.md.
Dense evidence never enters that window. Perception, verification, monitoring, and memory consolidation run as Sub Agents in isolated contexts and return structured results. Skills are typed by function, not by model family:
Failures do not trigger an immediate global replan. A Verification Sub Agent returns a typed report; the Main Agent writes a short textual diagnosis (the Reflexion pattern) and pushes a local recovery subgoal onto a Branch Stack. The stack top runs first. Success pops the branch and resumes the interrupted plan; about two consecutive failures promote the problem to high-level replanning. When the context hits a token cap, a Memory Integration Agent appends a chronological summary to HISTORY.md and merges revisable facts into MEMORY.md. All four external stores reset every episode. The reported scores do not use cross-episode learning.
Backends are official checkpoints with no extra fine-tuning. On the LIBERO family, Operational skills are π0.5 or Cosmos-Policy-LIBERO-Predict2-2B. Cosmos also acts as Imagination: five candidate chunks of horizon 16, scored by a lightweight vision-language model, greedy pick, about 1.2 seconds extra per selection. The planner gets five extra fixed cameras that the low-level policy never sees. Perception uses SAM3 masks and VGGT geometry. Each benchmark has its own AGENTS.md; the planner and primitive set stay fixed.
Standard LIBERO is already near ceiling. The orchestration gain shows up mainly on the long-horizon suite.
| Method | Goal | Spatial | Object | Long | Avg |
| π0.5 | 98.0 | 98.8 | 98.2 | 92.4 | 96.8 |
| EMERGE (π0.5, no world model) | 98.2 | 99.2 | 100 | 97.8 | 98.8 |
| Cosmos Policy | 98.2 | 98.1 | 100 | 97.6 | 98.5 |
| EMERGE (with world model) | 99.2 | 99.0 | 100 | 98.6 | 99.2 |
Perturbations and implicit instructions are where the gap opens. On seven LIBERO-Plus perturbation types, the world-model variant averages 93.9% against Cosmos Policy at 82.2% (+11.7). The no-world-model variant reaches 88.3% against π0.5 at 85.7% (+2.6), but drops on background (-4.9), sensor (-5.5), and lighting (-4.5) relative to π0.5. RoboHarness reports 93.2%. The paper states that extra cameras and a looser step budget are not matched to published baselines.
LIBERO-Pro implicit instructions move farther. Versus π0.5, Spat-S goes from 20.0% to 95.7%, Obj-S from 17.0% to 100%, L10-S from 8.0% to 59.1%. Combined explicit/implicit totals are 72.2/77.8 against 52.8/20.8.
On bimanual RoboDojo the average score/success is 20.46/13.54%, essentially tied with GalaxeaVLA G0.5 at 20.23/14.88%. Memory is 25.00/22.51% against G0.5 at 8.61/7.33% and π0.5 at 5.78/4.56%. Precision and long-horizon lag G0.5.
The real-robot task is a three-tier paper-cup pyramid (3-2-1). One hundred standard trials hit 94%. Four disturbance settings, 30 trials each: camera pose and count cut success to 85%; tearing down a partial stack and replanning holds 86%; cup diameter changes of 5% to 15% hold 92%; scene color stays at 93% to 94%. Task-specific priors cut planning steps 24.7% and wall-clock time 16.3%, with success 96.7% versus 95.2% without priors.
LIBERO-Plus ablations: drop Evaluation and the average falls to 85.6% (-8.3); drop the world model, 88.3% (-5.6); drop verification and monitor Sub Agents, 91.2% (-2.7). Sequential stripping goes 93.9 to 88.3 to 84.4 to 82.2, the last number matching Cosmos itself. Scoring only the current observation, with Imagination off, averages 85.1% under perturbations.
For teams already running π0.5 or Cosmos, this is a shell that does not touch weights. Verification, local recovery, and world-model action selection sit on top of released checkpoints. The jumps on perturbations and implicit language point to object identity, completion checking, and local retry as the actual bottlenecks, not another point of policy training.
This is incremental systems work, not a new action generator. The 0.7 to 2.0 point lift on standard LIBERO is, as the paper says, too close to ceiling to treat as a breakthrough. The reusable piece is the interface: skills registered by function, Sub Agents returning structured evidence, failures going onto a stack instead of a global replan. The bill is a closed-source planner, extra cameras, a longer step budget, and about 1.2 seconds per imagination step.
The authors flag the ceiling effect on standard LIBERO. Comparing the full system to an end-to-end policy also does not isolate the Sub-Agent workflow; that needs a matched single-agent harness with the same cameras, tools, prompts, and interaction budget, which this paper does not run. Real-robot claims depend on trial counts, intervention rules, and failure categories; the conclusion lists those as future reporting work.
The protocol is biased. Five extra planner cameras and a relaxed step limit mean published baselines may not have had the same budget. The high-level planner is closed-source Codex, so reproduction is tied to that choice. Memory does not persist across episodes. Per-benchmark AGENTS.md files and hand-written priors change the trajectory shape, so part of the efficiency gain is prompt engineering. On RoboDojo, memory and open-ended scores rise; precision and long-horizon do not, so this is not a blanket win. SAM3 still misses under occlusion, lighting shifts, and ambiguous appearance; VGGT is the patch.