New paper quantifies VLA sample efficiency: π0.5 needs the fewest demonstrations
The paper "How Many Demonstrations Does a VLA Need?" systematically studies the sample efficiency of vision-language-action models (VLAs): how many demonstrations and training steps are needed to reach a usable success rate on a new task. The authors fine-tuned and benchmarked 6 models — Physical Intelligence's π0 and π0.5, plus GR00T N1 through N1.7 — across 4 LIBERO suites, delivering quantitative answers.
Confirmed
- Systematic evaluation of the fine-tuning sample efficiency of 6 VLA models (π0, π0.5, GR00T N1 to N1.7) on 4 LIBERO tasks
- π0.5 is the most sample-efficient across all four LIBERO tasks: it needs 2-5x fewer demonstrations than GR00T N1.7 to reach the same or higher success rates, and also 2-5x fewer than π0
- On LIBERO-Long, π0.5 hits a 64% success rate with just 25 demonstrations, beating GR00T N1.7's 52% with 100 demonstrations
- With enough demonstrations, all VLAs achieve relatively high success rates
Unconfirmed
- Author rahulshivchand acknowledged in replies that the evaluation was done only on the LIBERO simulation benchmark; validation on other simulation tasks and real robots is still ongoing
Why it matters
- Collecting demonstrations is expensive, and sample efficiency directly determines the cost of deploying VLAs to new tasks; this work offers the first quantitative comparison of "how many demonstrations are enough," showing that continuously improved models like π0.5 have a generational edge in data efficiency
- However, the conclusions are currently limited to simulation, and whether they transfer to real robots remains to be verified
2026-11-11 ~ 2026-11-11 · 9 related posts
Primary sources
- [source] New paper benchmarks pi vs GR00T: how many demos does a VLA actually need to learn a task? — rahulshivchand · 2026-11-11
- π₀.₅ hits 64% on LIBERO-Long with just 25 demos vs GR00T's 52% with 100 — rahulshivchand · 2026-11-11
- Stronger zero-shot ≠ faster adaptation: GR00T N1.5 beats N1.7 in fine-tuning — rahulshivchand · 2026-11-11
- Ablation: freezing the VLM hurts π models far more than GR00T — rahulshivchand · 2026-11-11
- VLA Finetuning Study: π0.5 Needs 2-5x Fewer Demos Than π0 — rahulshivchand · 2026-11-11
- Freezing the VLM costs π models 23-56 points but barely hurts GR00T — rahulshivchand · 2026-11-11
- [source] Robot learning paper so far validated only on LIBERO; sim and real-world runs underway — rahulshivchand · 2026-11-11
2 near-duplicate retellings: k7agar · rahulshivchand