Allen AI's molmoact2 sim_eval runs zero-shot robot policy evals in ManiSkill
DJiafei · x · 2026-09-06
A post highlights allenai/molmoact2/simeval, an open-source setup for zero-shot evaluation of MolmoAct2 robot policies inside ManiSkill simulation, noting the results can be turned into demo videos via an LLM harness.
Key points:
- Unified CLI entry (runeval.py) with HTTP clients (DroidClient/YAMClient) calling the /act endpoint
- Supports Franka FR3 + Robotiq gripper (DROID) and bimanual YAM arm configurations
- Ships tasks like single-arm and bimanual 'put everything in box'
- Dependencies via uv; robot URDF/MJCF assets fetched through a download script
A ready-made scaffold for developers building robot policy simulation evals.
More from Embodied
- HKU's TILDE Distills Embodied Foundation Models into Robot Policies — hkuhk · 2026-10-23
- 10 of 12 Beta Robots Worked Without Fine-Tuning — Do We Still Need It? — micoolcho · 2026-10-23
- EmbodiedEvo lets robot agents self-train, banking reusable Skills and Lessons across tasks — jiqizhixin · 2026-10-23
- KAIST's FRED lifts robot policy success from 19.4% to 50% via self-distilled future video representations — kaist-ai · 2026-10-23
- GALAX Launches World's First RTX 5080 With Dual 12V-2x6 Connectors at $3000 — chemist_slime · 2026-10-23
- Figure prints 81 minutes of robot data per second, spent ~$8.1M on data last week — joecole · 2026-10-23