Alibaba's Qwen team launches RecreationBench to test hybrid computer-use agents by app recreation
TianbaoX · x · 2026-09-22
Alibaba's Qwen team introduced RecreationWorld / RecreationBench, a scalable, verifiable environment for evaluating hybrid computer-use agents that dynamically interleave CLI and GUI actions.
- Core idea: an agent must recreate an application purely by using it—testing exploration and execution—with programmatically verifiable scoring.
- Scale: 250 held-out applications across five native platforms (Ubuntu, macOS, Windows, Android, Web), scored on Prog (functional) and Visual channels.
- Leaderboard: the site compares models on Prog/Visual scores, threshold coverage (Prog ≥90% and =100%), and estimated per-task API cost, including models like GPT-6 Astra across platforms.
- The paper adds analysis of cost-vs-score tradeoffs, exploration behavior, and failure boundaries; code and website are public.
Related event: Alibaba's Qwen Team Releases RecreationBench for Hybrid Computer-Use Agents(2 posts)→
More from coding & agent
- Photoroom reclaims ~1,000 idle GPU hours a week with SkyPilot across Slurm and Kubernetes — skypilot_org · 2026-11-12
- Indie Britpop AI music video built with Nano Banana Pro, Kling, and free lip-sync via WAN2GP + LTX 2.5 — cocktailpeanut · 2026-11-12
- Cloudflare Talk: Swapping Some Agent LLM Calls for Fast Models Like Jev Gives Big Speedups — michellechen · 2026-11-12
- Engineer Curates 15 AI Agent Design Patterns Across Single-Agent, Multi-Agent and Feedback Tiers — MaryamMiradi · 2026-11-12
- Google Antigravity rolls out browser use, closing the loop from research to coding to live verification — rseroter · 2026-11-12
- Prettier 3.10 ships new Flow/JS syntax and a Zig-powered Yuku plugin — Vjeux · 2026-11-12