SoftwareGym synthesizes 182 app environments, lifting CUA OSWorld Pass@3 to 56.97%
Tian Jin · hf · 2026-10-12
Researchers from Shanghai Jiao Tong and Zhejiang University introduce SoftwareGym, an automated pipeline for synthesizing diverse GUI software environments to train computer-use agents (CUAs) at scale.
- Motivation: Native app environments are costly to build and invite overfitting; browser-based setups lack interaction depth.
- Method: Generate structurally diverse app designs, realize them as runnable applications, and verify functionality via GUI evaluation with iterative repair.
- Results: 182 synthesized apps across 12 categories with a 91.5% mean GUI task pass rate; a 120-task SoftwareGym-Bench evaluation suite.
- Training gains: A CUA trained on synthesized trajectories improves Pass@3 on the strictly out-of-domain OSWorld benchmark from 41.39% to 56.97%, and scores 40.0% on SoftwareGym-Bench.
Takeaway: synthesized software environments are a scalable, transferable supervision source for CUA training.
More from coding & agent
- Work2Skill Turns Human GUI Demos into Reusable Agent Skills, Beating No-Skill Baselines — nex-agi · 2026-10-12
- UCSD paper bakes theory of mind into RL reward, letting a 7B agent model partners solo — rohanpaul_ai · 2026-10-12
- Indie maker shows Claude Code prompt that builds SEO free tools, driving 4M impressions — tibo_maker · 2026-10-12
- Codex and ChatGPT Work Hit 40M Weekly Active Users, On Track for 100M by March 2027 — FinanceYF5 · 2026-10-12
- Two misconceptions in the REA era: software isn't dead, coding is — AlchainHust · 2026-10-12
- Cloudflare open-sources Clef decision models, tops Jev Decision Index — emmanuelvivier · 2026-10-12