Allen AI's molmoact2 sim_eval runs zero-shot robot policy evals in ManiSkill
DJiafei · x · 2026-09-06
A post highlights allenai/molmoact2/simeval, an open-source setup for zero-shot evaluation of MolmoAct2 robot policies inside ManiSkill simulation, noting the results can be turned into demo videos via an LLM harness.
Key points:
- Unified CLI entry (runeval.py) with HTTP clients (DroidClient/YAMClient) calling the /act endpoint
- Supports Franka FR3 + Robotiq gripper (DROID) and bimanual YAM arm configurations
- Ships tasks like single-arm and bimanual 'put everything in box'
- Dependencies via uv; robot URDF/MJCF assets fetched through a download script
A ready-made scaffold for developers building robot policy simulation evals.
More from Embodied
- Dev gets Google Astra running on the AYN Thor handheld with a dual-screen experience — Dimillian · 2026-09-06
- Cybercab rider logs 3 hours for $92.51, half the Uber cost, finds no major FSD V15 flaws — dean_rie · 2026-09-06
- Robot dog project Biscotti ships Simmy v2: train in sim, deploy to reality — adamamcbride · 2026-09-06
- 10 lidar scans + 120 photos: Blender + Astra builds a stunning 3D garden model — remilouf · 2026-09-06
- GPT Astra controls robot to clean a table, full chain-of-thought shown — Wonderful_Buffalo_32 · 2026-09-06
- Booster Robotics' T2 humanoid faces humans in a dance-off demo — DennisHongRobot · 2026-09-06