OpenAI data shows agent success collapses from 86% to 16% as task length grows
rohanpaul_ai · x · 2026-09-07
Rohan Paul argues "time horizon" is becoming a more useful measure of agent capability. Per OpenAI data, success without human intervention drops from 86% on sub-15-minute tasks to 16% on the longest 64–128-hour bucket, while intervention-requiring runs become common. Key points:
- Models can be strong locally yet unreliable over long execution trajectories
- Not tokens or benchmark scores — how long can a system keep useful control before a human intervenes?
- He predicts a benchmark like "human minutes consumed per completed task"
Related event: OpenAI models now handle 4.7-hour tasks at P50(4 posts)→
More from coding & agent
- 35 CVEs tied to AI coding tools this year, with no review budget to match — YvesMulkers · 2026-10-23
- Indie dev's agent tracks 519 directories, 311 submitted for SEO backlinks — fekdaoui · 2026-10-23
- ScienceClaw Hackathon at MIT Media Lab to build decentralized AI agent collectives for science — ProfBuehlerMIT · 2026-10-23
- What should a native agent harness handle beyond tool calling? — Glittering812 · 2026-10-23
- Running ~100 DeepSeek v4.1 flash agents in parallel for under $20/month — zack_overflow · 2026-10-23
- MCP's stateless 2026-07-28 spec drops session-id — how are stateful tools adapting? — Stock-Pumpkin-8859 · 2026-10-23