Memory and Reasoning Remain LLM Territory: Fable Beats Dream-1 64 to 9 on Cup Task
paigeinsf · x · 2026-11-19
Part three of the robot eval series covers memory and reasoning, where Fable leads Dream-1 by a wide margin.
- Find blocks hidden under identical cups, in order: Fable 64, Dream-1 9.
- Sort objects into three bins by category: Fable 62.5, Dream-1 10.
- Dream-1's #1 failure mode is going for the wrong target (wrong bin, peg, outlet, or button count), occurring in 45 of 100 runs.
The author's conclusion: LLMs won't replace robot models anytime soon — the best robot brain will likely use both.
More from Embodied
- SE3 Labs Releases A Wider Dream Eval, Opens 500-Episode Robot Demo Dataset on Hugging Face — paigeinsf · 2026-11-19
- LLMs zero-shot beat fine-tuned world action models on 7 of 10 robot tasks — paigeinsf · 2026-11-19
- The Secret Wasn't a Bigger Model: 500 Randomized Demos Lift Dream-1 Score 159% — paigeinsf · 2026-11-19
- Fine-tuned World Action Model Dream-1 Jumps 159%, Beats Frontier LLMs on Robot Tasks — paigeinsf · 2026-11-19
- Fine-tuned Dream-1 world action model jumps 159% on real-robot benchmark at 1/100th the cost — paigeinsf · 2026-11-19
- Robotics founder: "sim2real is BS" — labs buying data is a flash in the pan — Scobleizer · 2026-11-19