iOSWorld benchmark tests phone agents across 26 interconnected iOS apps with persistent user identity
dan_fried · x · 2026-10-08
OSWorld: A Benchmark for Personally Intelligent Phone Agents, presented at COLM 2026 by Lawrence Jang and Jing Yu Koh, is an interactive benchmark built around a persistent user identity spanning 26 custom iOS apps — analogs of OpenTable, Uber, DoorDash, Airbnb, and Chase.
The apps contain richly interconnected data (messages, transactions, travel histories, social relationships, preferences) to test how well phone agents operate in realistic personal contexts. Paper and code are available.
Related event: iOSWorld Benchmark Debuts at COLM 2026 for Personalized Smartphone Agents(2 posts)→
More from coding & agent
- HeyGen MCP turns Grok bot into your personal AI news anchor — HeyGen · 2026-10-08
- Canyon Overdrive lands on Spawn: an AI-built shooter with a full agent-first API — AIandDesign · 2026-10-08
- 8 Claude prompts to replace your Canva and Figma design subscriptions — Aiden_Tech_Ai · 2026-10-08
- Gemini CLI ships v0.65.0 nightly: OAuth loop fix, custom OTLP headers — gemini-cli-robot · 2026-10-08
- Claude Code lead says one prompt built a working app with zero bugs so far — bcherny · 2026-10-08
- Open-source plugin adds 29 UI languages, 73,312 entries, to DeepSeek Harness desktop app — seeweed7 · 2026-10-08