TerminalWorld: Top AI Agents Pass Only 62.5% of Real Terminal Tasks
jiqizhixin · x · 2026-07-03
UCL, Nanjing University, and Tencent jointly introduced TerminalWorld, a benchmark of 1,530 tasks across 18 workflows built by reverse-engineering real terminal recordings. The current optimal agent achieved a pass rate of only 62.5%, and its scores showed extremely low correlation with existing benchmarks, exposing the capability bottlenecks of AI agents in real terminal environments. The paper is available on arXiv, with code and the project page open-sourced.
More from coding & agent
- Reticle builds a verification layer that lets AI coding agents check their own generated code — Scobleizer · 2026-10-27
- Dev training a guard model to block prompt injection, tool and memory poisoning — gnukeith · 2026-10-27
- Dev trains a ~4GB-VRAM sentinel model to guard against prompt injection and poisoned memory — gnukeith · 2026-10-27
- Ex-Big Tech managers may thrive in the AI era: manage agents like you managed teams — lxfater · 2026-10-27
- SF 'Code Engineering' event picks up where 1,000-person harness engineering meetup left off — LangChain · 2026-10-27
- Prompt turns Claude Code into a forensics tool finding deleted iPhone photos in backups — alex_verem · 2026-10-27