OpenAI internal data: over 80% of successful 32-hour agent tasks still needed human intervention
DataLearnerAI · reddit · 2026-09-07
A successful 6-hour agent task isn't 6 hours of autonomy
The author tabulated data from OpenAI's internal report Research acceleration: The view inside OpenAI, which tracked real coding-agent tasks delegated by OpenAI researchers, grouped by estimated human task time, and added a new metric: among successful runs, how many needed at least one human intervention.
| Human task time | Success rate | Successful runs needing intervention |
|---|---|---|
| <15m | 94% | 8.5% |
| 1–2h | 90% | 32.2% |
| 4–8h | 88% | 51.1% |
| 16–32h | 82% | 72.0% |
| 32–64h | 76% | 82.9% |
| 64–128h | 67% | 76.1% |
Key insight: success rates stay high for a long while, but the nature of "success" shifts sharply — over half of successful 4–8h runs still required human intervention, and over 80% for 32–64h tasks.
The author argues:
- "An agent can complete a 32-hour task" ≠ "it can work autonomously for 32 hours."
- Agent evals should report task horizon × success rate × intervention rate, not just task completion.
- Unlike METR's controlled benchmark, this reflects real internal research workflows.
More from coding & agent
- charlie Marsh: rewrites beat external deps in the AI era — mohamedmansour · 2026-10-26
- OpenAI and Twitch launch Codex hackathon with $25,000 in prizes — OpenAIDevs · 2026-10-26
- Filtering secrets on the write path: why agent memory deserves more suspicion at ingest than retrieval — Classic-Being-7960 · 2026-10-26
- INT21's AI agent swarm rewrote Drama 3's inference runtime in Rust within one hour, beating SGLang-Omni — bingxu_ · 2026-10-26
- After OpenCode key leak: 5 practical rules for keeping keys away from agents — jacobholm_dev · 2026-10-26
- wandb 0.31.0 ships with 2x faster startup, adds JSON output for coding agents — wandb · 2026-10-26