OpenAI internal data: over 80% of successful 32-hour agent tasks still needed human intervention
DataLearnerAI · reddit · 2026-09-07
A successful 6-hour agent task isn't 6 hours of autonomy
The author tabulated data from OpenAI's internal report Research acceleration: The view inside OpenAI, which tracked real coding-agent tasks delegated by OpenAI researchers, grouped by estimated human task time, and added a new metric: among successful runs, how many needed at least one human intervention.
| Human task time | Success rate | Successful runs needing intervention |
|---|---|---|
| <15m | 94% | 8.5% |
| 1–2h | 90% | 32.2% |
| 4–8h | 88% | 51.1% |
| 16–32h | 82% | 72.0% |
| 32–64h | 76% | 82.9% |
| 64–128h | 67% | 76.1% |
Key insight: success rates stay high for a long while, but the nature of "success" shifts sharply — over half of successful 4–8h runs still required human intervention, and over 80% for 32–64h tasks.
The author argues:
- "An agent can complete a 32-hour task" ≠ "it can work autonomously for 32 hours."
- Agent evals should report task horizon × success rate × intervention rate, not just task completion.
- Unlike METR's controlled benchmark, this reflects real internal research workflows.
Related event: OpenAI Data: Most Long-Hour Agent "Successes" Required Human Help(2 posts)→
More from coding & agent
- Claude Code Users Press Anthropic on Whether Usage Resets Are Still Manual or Automatic — burhop · 2026-09-07
- Devtoolsniff bundles cursor rules generators, MCP configs, and an AI IDE cost calculator — Ice-Medium · 2026-09-07
- Blogger says Google Astra's natural tone broke his last dependency on Claude — StewartalsopIII · 2026-09-07
- AI agent sorts 15,000 files, 500GB Google Drive in 15 minutes with zero deletes — thisiskp_ · 2026-09-07
- One prompt built a neon two-player Pong game deployed to Apple TV in minutes — msg · 2026-09-07
- Our support bot has no memory, on purpose — a five-file agent architecture — Pitiful-Surround-285 · 2026-09-07