Dream-1 robot model eval: zero of 100 real-world episodes complete full tasks, mean score 8/100
paigeinsf · x · 2026-11-03
SE3 Labs ran the closed-source Dream-1 API zero-shot on a real bimanual robot across 100 episodes of RoboDojo – Real, covering generalization, memory, precision, long-horizon and instruction-conditioned tasks with fresh layouts each episode and a 180-second limit.
- No episode completed a full task; mean milestone score was 8/100, with 57 episodes reaching at least one milestone. Half stalled to timeout; the rest failed on grasps, placements, or wrong object/target.
- Zero-shot skills are real: Dream grasps, carries and places everyday objects and sometimes recovers from its own mistakes.
- Spatial bias is strong: in pepper sorting, success was 64% (9/14) when peppers started near the table center but only 19% (3/16) from the edges; the same pattern appeared in ring stacking.
- Instruction following is the biggest weakness — the model often executes the right motion toward the wrong goal.
The authors conclude the skills are genuine but bounded by training-data diversity, and plan to fine-tune Dream-1 and π0.5 to see how much task-specific data closes the gap. Full analysis and all 100 episodes are published.
Related event: Dream-1 Robot Model Tested Zero-Shot: Real Skills, Full Tasks Elusive(4 posts)→
More from Embodied
- Bleu Robotics raises €5M pre-seed to deploy humanoids in factories with just 10 demos per task — lukas_m_ziegler · 2026-11-03
- Machenta exits stealth with $40M seed, largest ever for a UK robotics startup — lukas_m_ziegler · 2026-11-03
- World model that predicts the future directly in 3D, fully open-sourced — YunzhuLiYZ · 2026-11-03
- Dream-1 zero-shot robot test: 100 public episodes show real skills but poor instruction following — paigeinsf · 2026-11-03
- ANYbotics ships ANYmal X, the first legged robot certified for explosive atmospheres — lukas_m_ziegler · 2026-11-03
- Tesla starts production-line Optimus build at Fremont, external deliveries eyed for late 2027 — lukas_m_ziegler · 2026-11-03