Energy-Based Policy Orchestration
Weirui Ye, Yen-Jen Wang, Elvis Hsieh, Zhiyang Dou, Shibo Zhao, Pieter Abbeel, Yilun Du, Leslie Pack Kaelbling, Tomás Lozano-Pérez
CoRL 2026
cs.RO, cs.LG
2026-11-02
EPO learns a per-policy energy over goals for frozen humanoid policies, refining commands and arbitrating at runtime: 7x fewer switches than MoE, deployed on a real Unitree G1.
Hierarchical control splits robot behavior into a high-level planner that issues task targets and low-level controllers that execute them, with a fixed map in between translating target into command. That map fails in dynamic tasks. A humanoid carrying a box may need to slow down, widen its stance, or pause precise tracking to recover from a slip or an unexpected contact, and those physical changes happen on a millisecond scale, far faster than the high level updates. A command that matches the target kinematically can be physically infeasible.
Once you keep a family of low-level policies around, some tuned for precision and some for disturbance recovery, execution becomes two coupled decisions: which policy to run, and what goal to hand it. The best goal depends on which policy pursues it; the best policy depends on which refined goal stays feasible. The paper names this problem policy orchestration and proposes EPO. The team spans MIT, Berkeley, CMU, Harvard, and Amazon; it appears at CoRL 2026.
EPO is a runtime layer between the planner and a frozen family of goal-conditioned policies. For each policy it learns an energy function (an energy-based model: lower energy means more plausible, and inference searches for low-energy points):
Energies are trained per policy but against shared definitions of feasibility and consistency, so they compare across policies. Adding a policy costs only its own energy models, about 30 GPU-minutes, with everything else untouched; an MoE gate needs a full retrain (40-60 GPU-minutes) on every library change.
Simulation runs in Isaac Lab on a Unitree G1, 384 episodes per setting across flat, rough, and stair terrain, with a 16D command interface (base velocity, torso height, end-effector pose):
| Method | Metric | Result |
| EPO-M2 vs general policy | tracking error | 0.542 vs 0.568 |
| EPO-M4 vs MoE-M4 | standing manipulation error | 0.420 vs 0.461 |
| EPO-M4 vs MoE-M4 | switches per episode | 2.08 vs 15.46 |
| consistency-only ablation | push survival (loco) | 78.9% (full: 96.4%) |
| adding a policy | GPU cost | 30 min vs 40-60 retrain |
MoE-M2 never leaves the general policy in 56% of episodes; the router barely routes. Under pushes strong enough to topple the standing specialist, EPO-M4 survives 97%. Ablations attribute nearly all robustness to the feasibility signal (dropping it cuts standing push survival from 95.3% to 58.6%) and precision to goal refinement (5-6% lower error on base velocity and torso height, 25% lower on torso height after a push). Refinement is also most of the compute: without it the orchestration ceiling goes from 83 to 615 Hz, and full EPO-M4 takes 24.3 ms per cycle on an RTX PRO 6000. Under a 3 m/s push it switches to the robust policy within 40 ms and returns 0.48 s later.
Hardware results are demonstrations: the same frozen policies and energy models run on a real G1 with no fine-tuning for teleoperated box pick-up, recovery from a basketball strike, and door opening. In a 5-trial standing-push test, EPO-M2 recovers in all five; the standing specialist alone in four.
Teams building humanoid whole-body control tend to accumulate a drawer of policies and lack an execution layer that combines them without retraining. EPO lets precision-oriented and robustness-oriented controllers train separately and arbitrate online by energy score, and adding a skill degenerates to a 30-minute offline training run. Versus MPC-style online optimization it is much cheaper, optimizing only at the goal interface; versus MoE it switches an order of magnitude less often, and constant switching is itself disruptive on hardware. Energy as a common currency across policies is not specific to humanoids and travels well.
The authors list three: learned energies may be unreliable under distribution shift, poorly covered goals, or miscalibration for newly added policies; runtime cost grows with the number of candidates, dominated by gradient refinement; and EPO sits at the policy-goal interface, replacing neither action-level safety filters nor formal constraints.
Two more caveats from the read. The precision gains are small, about 4.6% relative in the main comparison (0.568 to 0.542), and EPO's weighted survival sits just below the more robust reference (96.8% vs 97.7%); what it buys is having both at once, not a new frontier on either axis. The hardware section is demonstrations plus one 5-trial test with no quantitative control, so 'deploys on real hardware' reads as feasibility rather than a benchmark. All energy models are trained within one simulation protocol; whether an external policy library drops into the same energy definitions is untested.