GPT-6 Astra drives a robot arm to paint the Golden Gate Bridge, scores 72.6% on OSWorld 2.0
APPSO · wechat · 2026-09-27
A Stanford student gave GPT-6 Astra a robot arm, a brush and a camera, and the model painted the Golden Gate Bridge itself — the clip drew 2M+ views and a reply from Sam Altman. Other viral demos: building a space shuttle in Blender in 10 minutes, beating all 48 levels of Neal Agarwal's captcha game, and mining a diamond in Minecraft overnight.
Technically, a Browserbase engineer's teardown explains Astra dropped the screenshot-plus-pixel-click approach: it reads the Accessibility Tree for semantic UI understanding (with screenshots for visual detail), and runs a "code mode" — a persistent Node REPL where it writes Playwright/PyAutoGUI scripts to control browser and desktop, re-reading the screen after each step. A Guardian Policy model classifies risk per action, pausing for human approval on high-risk moves. On OSWorld 2.0 it scores 72.6% (vs 65.7% prior gen) with average task time cut from 75 to 40 minutes.
Greg Brockman calls ComputerUse a qualitative leap and "almost a universal connector." On the physical side, Astra zero-shots robot arm pose commands in RoboCurve, succeeding 19/20 on a block-into-bowl task versus 40% for Claude Fable 5.1.
More from coding & agent
- Reward hacking bugs revealed: unpruned git history let models peek at patches, edit tests in shared sandbox — willcb · 2026-09-27
- A Content Pinball Machine built entirely with Claude Opus 5.5 satirizes viral randomness — CurieuxExplorer · 2026-09-27
- Dev caches hide 264 CVE-laden packages and 71.7 GiB no one audits, warns Cache Commander dev — julsimon · 2026-09-27
- WebMCP could become the HTML/API layer of the agentic web — Thionne_WTZ · 2026-09-27
- Stanford/Together AI paper: agent teams hit 66.7% vs 48.8% for single agents — mark_k · 2026-09-27
- GLiNER2.5-Decide hits 7.6ms local routing calls on M5 Pro, an LLM-free alternative to typed JSON — ivan_digital · 2026-09-27