LLMs Begin Directly Controlling Robots
DorsaSadigh · x · 2026-07-15
The author notes spending the past year on LLM post-training, but is now witnessing a major leap: "LLMs are starting to directly control robots, collect data at scale, and automate numerous robotic processes."
The core insight comes from VIA: Visual Interface Agent for Robot Control. Researchers equipped agents like Claude Code and Codex with browser-based visual interfaces, allowing them to control robots using a mouse and keyboard via MCP tools, much like a human would—and it actually worked.
The key takeaway isn't just robot control; it's the integration of general-purpose agents, visual interfaces, and robotics, pointing toward the automation of future robot data collection and training pipelines.
More from Embodied
- Musk: Cybercab certified at 165 Wh/mi, the most efficient production EV ever — elonmusk · 2026-09-11
- MKBHD goes hands-on with the first folding iPhone; $2,000 48MP selfie cam mocked — alexmacgregor__ · 2026-09-11
- NTU spin-off Ropedia launches HOMIE Gen 2 wearable system to train robots from human experience — liuziwei7 · 2026-09-11
- M.2-Oculink eGPU Link Silently Downgrades to PCIe Gen1 — Here's How to Check — El_90 · 2026-09-11
- Replit Agent in a robot builds and publishes websites autonomously via MCP — amasad · 2026-09-11
- TARS Robotics unveils embodied foundation model AWE: 15+ tasks, one model, zero retraining — heyshrutimishra · 2026-09-11