Dev builds a screenshot-free computer use agent powered by the macOS accessibility API
jamesmurdza · x · 2026-09-18
Developer jamesmurdza released a computer use agent that works without screenshots: it uses the macOS accessibility API to interact directly with applications, sidestepping the usual vision-based screenshot-and-click loop.
The agent can run in the cloud via Daytona, connect to any LLM, and control any macOS app. The author demos it building a simple program in Xcode.
More from coding & agent
- Agent safety startup Raindrop raises to $50M total, launches Simulations to catch failures pre-production — ycombinator · 2026-09-18
- YC-backed Raindrop launches Simulations to catch AI agent failures pre-production — ycombinator · 2026-09-18
- RedMonk: Developers were the new kingmakers — agents are next in line — rseroter · 2026-09-18
- openwiki v0.5.2 adds bob coding agent integration, now 6 total — LangChain · 2026-09-18
- The 'Seniority Cliff': skipping junior-level friction may hollow out engineering intuition — Jumpy-Increase9337 · 2026-09-18
- Aident's First Skill Uses Agents to Submit Products to 30+ Directories at Once — alifcoder · 2026-09-18