Vercel's agent-desktop reads accessibility trees to automate desktop apps, 16.4k stars
victor_explore · x · 2026-11-05
Vercel Labs open-sourced agent-desktop, a computer-use CLI for AI agents that has hit 16.4k GitHub stars.
- Core idea: coding agents can drive browsers, but desktop automation still relies on fragile screenshot-clicking. agent-desktop instead reads the OS accessibility tree and returns stable element refs, letting agents act on refs, re-read the UI, and verify — an observe-act-verify loop.
- Capabilities: 56 operational commands, macOS/Windows/Linux support, ships as CLI and C library, MCP support, built in Rust, Apache-2.0.
- Positioning: a tool agents call, not an agent — no embedded model, no GUI. It only automates desktop apps exposing accessibility trees; web content goes through agent-browser, with a DevTools-port bridge for Chromium/Electron apps.
More from coding & agent
- Hugging Face ships safeagents scrub/publish to redact shared agent trajectories — yb2698 · 2026-11-06
- Hugging Face open-sources safeagents, a local proxy that blocks reward-hacking tool calls — yb2698 · 2026-11-06
- Synthesizing RL Envs from Open Trajectories: LFM2.5-24B Self-Scores Task Difficulty — yb2698 · 2026-11-06
- Hugging Face livestreams open RL training: trajectory-built envs, safeagents, LoRA evals — yb2698 · 2026-11-06
- Hugging Face synthesizes RL envs from open trajectories: Repair, Build, Extend, QA — yb2698 · 2026-11-06
- A 7-step blueprint for building AI agents, with a 10-framework comparison chart — mdancho84 · 2026-11-06