Qwen releases RecreationBench: 250 tasks testing agents that rebuild real apps
burny_tech · x · 2026-09-21
Qwen released RecreationBench on Hugging Face: a 250-task benchmark spanning Ubuntu, macOS, Windows, Android and Web, designed to test hybrid computer-use agents that recreate real applications from scratch.
Unlike screenshot-and-click computer-use evals, it measures an agent's full understanding and reimplementation of real apps, offering a new yardstick for end-to-end software-building agents.
Related event: Alibaba's Qwen Team Releases RecreationBench for Hybrid Computer-Use Agents(2 posts)→
More from coding & agent
- User automates entire video clipping workflow with Nous Research's Hermes agent — Teknium · 2026-11-12
- As models improve, writing good evals gets harder and more important — amankhan · 2026-11-12
- Prime agent ships with day-one multi-agent communication, runs in Minecraft unmodified — sarahcat21 · 2026-11-12
- Autonomous agent edits just 1,024 weights to lift Qwen3.5 0.8B from 7.4% to 37.4% on MATH-500 — VictorSanh · 2026-11-12
- Omarchy 4.6 lets you share local AI models across your tailnet, says DHH — alexcovo_eth · 2026-11-12
- Coding harnesses hand shell access to inference providers — 4-step gateway mitigation — shensi · 2026-11-12