MobileWorld benchmark: best phone GUI agents score only ~52% on 201 tasks
East-Muffin-6472 · reddit · 2026-08-19
The MobileWorld paper proposes a new benchmark for autonomous mobile agents: 201 tasks across 20 apps (comms, messaging, productivity, etc., <5% system apps), with two novel evaluation axes:
- User Interaction tasks: key info is deliberately missing so the agent must ask the user (simulated by GPT-4);
- MCP tasks: tools like GitHub and arXiv let agents perform complex data gathering and actions impossible via GUI taps alone, replacing slow multi-step interactions.
The setup uses a planner-executor architecture: a VLM planner sees only screenshots (no accessibility tree) and outputs natural-language actions, which a grounding model converts to precise (x,y) coordinates.
Results: the best combo, Gemini-3-Pro + UI-Inst-7B, averages 52%, end-to-end GUI models do far worse, and failures concentrate on the two new axes — proactive clarification and tool use remain clear weaknesses for phone agents.
More from coding & agent
- Sub8 enables local Linux automation controlled by Claude or local models — Daniel_Farinax · 2026-08-19
- Always-on AI Agents need background updates without restarts, hitting OS-level limits — koltregaskes · 2026-08-19
- Open Source MemMachine: Universal Memory Layer for AI Agents — tom_doerr · 2026-08-19
- Rust Web Framework Earendil Hits Hacker News Front Page — mitsuhiko · 2026-08-19
- Fix Hallucinations with Local Wikipedia, Not Bigger Models — PieBru · 2026-08-19
- Yeku: An MCP Server to Deploy Full Apps in One Message — itsAg3nt47 · 2026-08-19