LLMs Begin Directly Controlling Robots
DorsaSadigh · x · 2026-07-15
The author notes spending the past year on LLM post-training, but is now witnessing a major leap: "LLMs are starting to directly control robots, collect data at scale, and automate numerous robotic processes." The core insight comes from **VIA: Visual Interface Agent for Robot Control**. Researchers equipped agents like Claude Code and Codex with browser-based visual interfaces, allowing them to control robots using a mouse and keyboard via MCP tools, much like a human would—and it actually worked. The key takeaway isn't just robot control; it's the integration of general-purpose agents, visual interfaces, and robotics, pointing toward the automation of future robot data collection and training pipelines.
More from Embodied
- HarmoHOI generates multi-view hand-object videos and aligned 3D motion in one diffusion model — cn-scut · 2026-07-21
- Tesla is reportedly building a humanoid robot factory aimed at 10 million units a year — davidpattersonx · 2026-07-21
- Anthropic is reportedly in talks to buy Physical Intelligence — MarvinTBaumann · 2026-07-21
- IROS 2026 workshop sets Aug. 10 paper deadline for embodied world models and WorldArena 2.0 — 量子位 · 2026-07-21
- Work Louder’s new input device draws praise for its multi-input design — Dimillian · 2026-07-21
- Sudo Tech shows 10+ embodied AI skills at WAIC and ties them to a real CATL line — 机器之心 · 2026-07-21