Computer use is the best lens for LLM robotics: from coordinates to intuition
Scobleizer · x · 2026-11-18
The author argues computer use is the closest LLM capability to robotics, since both run the same loop: observe, reason, act, adjust. Key takes:
- Robotics may repeat computer use's evolution: early computer use had models output "click at (x, y)", which was slow and weak; scaling took off when models wrote short code to batch actions in one go. Physical skills may similarly shift from coordinates to intuition.
- A robot is the same agent already strong in text, stretched toward vision and the physical world, with vision grounding steadily improving inside LLMs.
- The real debate is how physical skill gets learned: the old answer was behavior cloning plus massive real-world interaction; new paradigms are still contested.
More from AGI Musings
- Coding agents make software ideas horizontally transmitted — and more virulent — sebpaquet · 2026-11-18
- If AI automates engineering and research, moats shift to data, compute and capital — burny_tech · 2026-11-18
- Cyber consensus: exploits now land 7 days before patches, no end in sight — alliekmiller · 2026-11-18
- Mathematician Michael Goldman: trusts LLM reviews over human peer review — skdh · 2026-11-18
- Gary Marcus shares 'famous last words' table of Enron, Lehman, SVB, FTX CEOs to mock AI hype — GaryMarcus · 2026-11-18
- A 26-Patent Inventor Asks: Can Curiosity Be Taught in the AGI Era? — tctjr · 2026-11-18