ApprenticeBench: Top Models Now Beat APIs Through GUIs, the 'CUA Tax' Has Disappeared
ysu_nlp · x · 2026-09-11
Boyu Gou (SeeAct, UGround, Mind2Web 2) launched ApprenticeBench, the first benchmark for computer use and continual learning in a realistic accounting job. On a 100-task run combining learning and execution, Fable 5.1 scored 72% via GUI vs 70% via API, and GPT-6 Astra 68% vs 65%—meaning the 'CUA tax' has disappeared for top models, with GUI-based agents now outperforming API workflows. He argues computer use has been seriously underestimated and evaluated too narrowly since Opus 4 and Sonnet 4.5.
More from coding & agent
- huggingface_hub 2.3.0 cuts 1M-file listings to 20s, adds live Bucket following — lhoestq · 2026-10-27
- Nous Research ships Advanced Delegation plugin letting Hermes Agent route subagents to different models — NousResearch · 2026-10-27
- A dentist with a 3D printer discovered OpenAI Codex — and never looked back — gabrielchua · 2026-10-27
- Stanford's PaperFormalizer turns research papers into Lean-verified versions with author in the loop — StanfordUniversity · 2026-10-27
- Hugging Face TRL makes score centering + masked importance sampling default in async GRPO — _lewtun · 2026-10-27
- a16z: agents follow tech's oldest law — chaos first, then rein it in — a16z Newsletter · 2026-10-27