3 Franka arms stack 9-block tower with Gemini Robotics-ER as VLA brain
Zergylord · x · 2026-09-19
An open-source project, IsaacSimGeminiRobotics, demonstrates multi-robot cooperation: three Franka FR3 arms build a 9-block tower inside NVIDIA Isaac Sim with ROS 2, orchestrated by Google Gemini Robotics-ER acting as a Vision-Language-Action (VLA) brain.
- Architecture: Gemini Robotics-ER analyzes overhead camera feeds, plans tasks, and dispatches parallel robot commands via Function Calling
- Dual modes: switch between the Gemini VLM orchestrator and a rule-based motion planner that needs no API
- Engineering insight: the model produces "ghost grasps" because it can't see a missing attach frame; gating tools on TF plus a precondition check eliminated them — "plan → execute → write the limit back into the machine spec"
- Monitoring: a Salabim-inspired React dashboard shows live KPI utilization, a 2D digital twin map, timelines, and discrete event logs
- Cross-OS: Windows ↔ WSL2 bridged via FastDDS Unicast, automated at boot
More from coding & agent
- Anthropic's Head of Product Drops a 28-Minute Masterclass on Agents in Production — ifioknkem · 2026-09-20
- Teknium: Jev can't compact context well — Hermes summarizes 95% of it away — Teknium · 2026-09-20
- HarnessRouter open-sources a unified API to run Codex, Claude Code and more as agent backends — daniel_mac8 · 2026-09-20
- HarnessRouter: routing agent harnesses instead of models, a fresh infra idea — daniel_mac8 · 2026-09-20
- MCP tool naming: short generic verbs vs explicit prefixes for LLM tool selection — skvark · 2026-09-20
- GameToMac launched 10 days ago and already runs AoE IV, CS2 and Diablo IV on Apple Silicon — nickbaumann_ · 2026-09-20