HKU open-sources PhoneGo: on-phone AI agent cuts steps 47%, latency 56% vs vision agents

aigclink · x · 2026-11-20

HKU's HKUDS lab open-sourced PhoneGo, a personal AI agent that runs directly on Android phones and executes cross-app tasks from natural-language instructions — no PC, no step-by-step screenshots. Give it a task like "find hotels near HKU on Xiaohongshu, then check prices on Ctrip" and it searches posts, extracts hotel names, switches apps, and queries dates and rates on its own, with apps hidden in a virtual screen so your phone stays usable.

The key idea is hybrid CLI+GUI execution: known app operations are invoked as pre-compiled commands, and a vision model only takes over on unfamiliar screens. Offline exploration builds an "App Navigation Map" of pages and paths compiled into executable commands; online, the agent parses intent and prefers the CLI route.

Compared with pure-vision phone agents, PhoneGo cuts operation steps by 47%, execution time by 56%, and token consumption by 55%. MIT-licensed on GitHub, already at 1.3k stars.

Related event: HKU Open-Sources PhoneGo, a Natural-Language Android Agent(2 posts)→

Original post →

More from coding & agent

coding & agent channel →