X-Planner: Event-Structured Task Planning for Embodied Intelligence
Howard Lu, Shalfun Li, Porter Pan, Cris, Lumen, Cyril, Eric Hu, Lily Li, Maeve Zhang, Robert Wang, KZ Zheng, Viggo Chen, Tim Ding, Regsis Cheng, YJ Xiao, Kian, Hai Lin, Alan Song, Elise Ma, Gody Li, Victor Yao, Yohann Tang, Ingrid Yu, Jason He, James Wang, Ryan Yu, Ping Yang, Chris Pan, Vincent Chen, Roy Gan, Hao Wang, Qian Wang
cs.AI
2026-09-22
X-Planner, an event-structured planning front-end, scores 71.60 and 53.75 Task Progress on two real-robot suites and 0.9011 BERTScore-F1 offline, second to kimi3.
Long-horizon manipulation needs an executable layer of sub-goals between a high-level instruction and low-level control. Most Vision-Language-Action (VLA) models bury that structure in a reactive map. Existing chain-of-thought planners often stop at task-level labels, or they serialize a long reasoning trace token by token.
X-Planner, from X Square Robot, is a planning front-end for a downstream world-action model. Supervision is supposed to land on event boundaries and execution errors. The representation exposes both a readable event state and a parallel latent chain of thought.
Planning data mix Ego, UMI, and teleoperation under four levels: L3 task, L2 subtask, L1 executable action, L0 finest temporal span. Teleoperation is labeled at all four levels. Ego and UMI keep L1–L3 because faster motion makes L0 boundaries unreliable. Takeover times supervise error recognition during execution; human-designed failure demos are added so the error distribution is not locked to the collecting policy. A deterministic analysis subset keeps 1,500 of 1,654 episodes, dropping 154 frequent redundant combinations, and still covers 167 datasets and 30 named task categories. In that subset, rigid objects are 58.2% of episodes; the median episode has five subtasks and lasts 37.1 s.
A Qwen-series vision-language backbone exposes two plan forms. The discrete interface emits compact JSON: an ordered initial plan at start; normalized progress, current and next events, continuation, and error flags during rollout; a media-boundary marker at the end. The latent interface uses Staircase Decoding. The Transformer is split at a relay depth Nr. Lower layers share visual-language grounding; upper layers update each reasoning position in parallel, producing Kc continuous thought states that enter downstream cross-attention directly. A frozen lightweight language model reconstructs a textual chain of thought from those latents as a semantic anchor, rather than cloning one autoregressive hidden trajectory.
The downstream world-action model stays frozen. The discrete path is trained with full-JSON supervised fine-tuning, plus an alignment loss that pulls planner text features toward the downstream model’s original text encoder. Real-robot evaluation uses event mode. The latent form is not compared.
Offline two-step planning ranks second of four models. BERTScore-F1 without baseline rescaling: X-Planner 0.9011, kimi3 0.9108, Qwen 0.8878, Doubao 0.8846. Doubao Judge Overall: 1.411 against 1.434 / 1.367 / 1.366. Both scores are text quality, not task success, and no significance test is reported.
Real-robot tests use tabletop bimanual arms and multi-view cameras. The metric is Task Progress, a dense 0–100 score. On the reasoning suite (sort headphones, classify by shape, press buttons in order, pair items, pick fruit by instruction) the averages are:
| System | Suite mean |
| X-Planner | 71.60 |
| U-Scratch | 59.50 |
| π0.5 | 56.40 |
| DreamZero | 32.70 |
| LingBot-VA | 31.60 |
That is 12.10 points above the strongest baseline, U-Scratch. Per-task scores are mixed: ordered button pressing 64% versus U-Scratch 18%, fruit picking 96%; U-Scratch leads shape classification 82% to 78%, and π0.5 leads pairing 77% to 36%. On the generalization suite (one cluttered table, changing instructions) the mean is 53.75, against DreamZero 28.50, π0.5 24.00, and U-Scratch 18.50. No trial-level scores and no error bars.
Treating a plan step as an action-grounded semantic event sits closer to a real handoff than a fixed-length chunk or an episode caption. A person can edit one event sentence; the downstream model still has to realize it. Sharing a backbone across discrete and latent interfaces is a way to split interpretability from parallel compute.
The robot lead is for the full event-mode system. Offline text still trails kimi3. If the goal is an inspectable, editable planning front-end, this is a working engineering path. If the goal is to price Staircase Decoding or designed failures on their own, the paper does not.
The authors are explicit: there is no ablation that isolates error supervision, the discrete form, the latent form, the staircase schedule, or any single training objective, and no end-to-end latency. A controlled comparison of the two plan forms is future work. Training conditions on annotated history and initial plans; deployment conditions on the model’s own rollouts, an oracle-context gap. Episode-end marks a synchronized media boundary, not task success. Evaluation stays on tabletop bimanual arms, with no mobile or humanoid platform.
The offline pool is four models and one Doubao judge. Whether BERTScore is a fair metric for planning text is not discussed. U-Scratch is a from-scratch instruction-to-action system with the same real-robot supervision, a system-level reference rather than a single-component ablation.