JarvisGUI benchmark tests GUI agents across Android, Windows and Ubuntu in one workflow
maier_ak · x · 2026-09-23
JarvisGUI, an EMNLP 2026 paper, introduces a cross-device GUI agent benchmark that runs Android, Windows and Ubuntu VMs inside Docker. It defines a typed I/O system, 118 atomic actions, and 150 composite workflows (442 subtasks), evaluated with Total Task Success and Sub-task Success metrics to test coordination, state-keeping and long-horizon planning.
Related event: JarvisGUI benchmark: open GUI agents struggle across devices(4 posts)→
More from coding & agent
- lattedit launches free local-first browser LaTeX editor with MCP support for coding agents — cneuralnetwork · 2026-11-11
- Rivet launches SQLite Forking: 20ms agent-thread forks with zero data copying — samgoodwin89 · 2026-11-11
- YouArt Desktop launches: AI agent edits video in CapCut, DaVinci and Premiere from client feedback — mhdfaran · 2026-11-11
- Boubou: one-command local voice AI that serves your phone and robots over Wi-Fi — LuozhuZhang · 2026-11-11
- Python 3.15 makes AttributeError actionable: suggests .append for .push mix-ups — KhuyenTran16 · 2026-11-11
- sentence-transformers maintainer releases new embedding pooling training method, code & checkpoints — tomaarsen · 2026-11-11