5 open-source repos for giving AI eyes and hands on your computer
udmrzn · x · 2026-10-28
A roundup of five GitHub repos that let AI see your screen and operate your apps:
- OmniParser — screen parsing
- Agent-S — computer-use agent
- UI-TARS Desktop — GUI agent desktop client
- OpenAdapt — workflow automation from recordings
- screen-use — screen operation capabilities
The takeaway: AI gets a lot more interesting when it can actually see your screen and operate the apps on it.
More from coding & agent
- Multimodal GraphRAG models missing and contradictory evidence for storm damage assessment — JeremyCMorgan · 2026-10-28
- Vibe coding in real life: 20 hours of personal-project work done in 2 — gleech · 2026-10-28
- LangChain open-sources Content Studio, a Slack design agent, and shares its harness design — BraceSproul · 2026-10-28
- "If AI can do it, humans shouldn't": team lets AI agent review PRs — Yuchenj_UW · 2026-10-28
- Sequoia, Citi and OpenAI launch No Limit: build a poker bot in 60 minutes, $100K grand prize — polynoamial · 2026-10-28
- Nyxis: a Lean framework with formally proven authorization safety for AI-written backends — udmrzn · 2026-10-28