Astra, Fable, Muse Reportedly Zero-Shot Robot Benchmarks, Sparking Academic Despair and Debate
Starting September 13, NYU robotics professor Lerrel Pinto repeatedly described deep despair across the robotics community: systems like Astra, Fable, and Muse were reportedly beating robotics and world-model benchmarks zero-shot. He framed this as a hard-to-swallow but necessary generational leap that academia must adapt to.
Confirmed
- Pinto and commentators relaying his posts (FinanceYF5, Scobleizer) all reported zero-shot results by Astra, Fable, and Muse on robotics and world-model benchmarks, which Pinto characterized as a step-change in capability.
- Researcher yoavgo pushed back: the new models did solve problems his team had long struggled with, but they were not zero-shot—they were trained intensively per task, disputing the 'zero-shot' label.
- Respondent Kostas Penn pushed back against the despair framing.
- Aviv Tamar noted that robotic touch remains unsolved and tactile training data is scarce, keeping it a worthwhile research direction.
Unconfirmed
- Whether these systems truly achieved benchmark results zero-shot remains disputed between Pinto's framing and yoavgo's objection about their training regime.
- None of the posts included specific benchmark scores, data, or sources for the claimed results.
Why it matters
- If zero-shot systems truly surpass benchmarks without task-specific training, the traditional paradigm of designing methods to score on benchmarks could be systematically overtaken, reshaping academic research priorities and evaluation.
- yoavgo's rebuttal cautions against declaring a paradigm shift before clarifying how these models were actually trained; the 'zero-shot' label may be overstated.
- Tamar's note points to a remaining gap: tactile sensing and related data are untouched by these systems and could be academia's next focus.
2026-09-13 ~ 2026-09-14 · 8 related posts
Primary sources
- Robotics researcher: Astra, Fable, Muse zero-shot benchmarks spark academic despair — LerrelPinto ·
- Robotics academics in despair as Astra, Fable and Muse zero-shot benchmarks, says NYU professor — LerrelPinto ·
- Yoav Goldberg: new robot models aren't zero-shot — they're trained to the max on each task — benno_krojer ·
- [source] Robotics researcher: Astra, Fable, Muse zero-shot benchmarks spark academic despair — LerrelPinto · 2026-09-13
- [source] Robotics academics in despair as Astra, Fable and Muse zero-shot benchmarks, says NYU professor — LerrelPinto · 2026-09-13
- Robotic touch remains unsolved with scarce training data, researchers push back on zero-shot hype — LerrelPinto · 2026-09-13
- [source] Yoav Goldberg: new robot models aren't zero-shot — they're trained to the max on each task — benno_krojer · 2026-09-13
- Blogger Claims Astra, Fable and Muse Zero-Shot Robot and World-Model Benchmarks — FinanceYF5 · 2026-09-14
3 near-duplicate retellings: LerrelPinto · Scobleizer · MikeShou1