HybridCUA mixes GUI with CLI commands, hits 53.6% on OSWorld (+14.8 pts)
Tongbo Chen · hf · 2026-09-30
The paper argues next-gen computer-use agents should combine GUI interactions with the CLI: GUI is general but slow and error-prone, while API/tool augmentation doesn't scale across apps.
- Key challenge: models don't know when or how to use the CLI. A data pipeline produces GUI-only, CLI-only, and interleaved trajectories, yielding HybridCUA-8K (5K hybrid trajectories + 3K verified RLVR tasks)
- Two-stage training: SFT on constructed trajectories, then RL with CLI-aware rewards encouraging selective, reliable CLI use
- HybridCUA-9B reaches 53.6% on OSWorld (+14.8 pts over base) and gains 4.0 pts on WindowsAgentArena, showing cross-platform generalization
More from coding & agent
- RSI Arena: 8 research agents autonomously train rival models for human judging — my_cat_can_code · 2026-09-30
- Auto-research loop on 120 B300s finds 40% Kimi K3 inference gain for $9,176 — bookwormengr · 2026-09-30
- LENINROOMS: Building an Infinite Soviet Apartment Backrooms With Claude — teortaxesTex · 2026-09-30
- dart-query MCP server adds DartQL bulk task operations to cut token usage — modelcontextprotocol · 2026-09-30
- ISIR MCP connector lets agents query Czech insolvency register by company or person — modelcontextprotocol · 2026-09-30
- New Yorker-style illustration Skill hits 400 stars, monetizes via Baidu agent ecosystem — oran_ge · 2026-09-30