Robot Policy Metadata Conditioning Fails: Text Prompt Completely Ignored
DominiqueCAPaul · x · 2026-09-30
The author tried to replicate physicalint's approach of adding metadata about the episode or action chunk to the text prompt for the π0.7 model, using speed as the conditioning variable: the dataset contained two recording sessions, one with double the speed of the other, and a suffix "speed: fast|slow" was appended to the prompt.
Result: the policy completely ignored the text prompt, and behavior did not change when switching between "slow" and "fast".
Further analysis showed the speed information could already be extracted from the images themselves — a logistic regression distinguished the two sessions on 99.8% of held-out frames. The only visible difference was a slightly different placement of the box and tray on the table, but that was apparently enough.
Takeaway: if the conditioning information is already implicit in the observation images, extra metadata conditioning may be ignored by the policy; check for such leakage before running these experiments.
More from Embodied
- NVIDIA's HumanoidMimicGen turns one teleop demo into thousands of humanoid demonstrations — AjayMandlekar · 2026-09-30
- Another glowing CyberCab ride: whurley shares second robotaxi experience — whurley · 2026-09-30
- Laser on a 3D printer turns Kapton tape into laser-induced graphene with PWM control — johnowhitaker · 2026-09-30
- Muse Charm Puts an AI Agent on Your Keychain — and It Feeds on Your Data — BenBajarin · 2026-09-30
- Chaining MiniMax H3 keyframe clips for an arcade racer without perspective drift — NoMarionberry9951 · 2026-09-30
- LeCun co-authors AD-E2E-JEPA: world model drives end-to-end without any driving policy — ylecun · 2026-09-30