AgentTime benchmark: agents' runtime control varies 2.4x, Astra sleeps to use its time budget
maksym_andr · x · 2026-10-12
The arXiv paper "AgentTime: Can Agents Estimate and Control Their Own Runtime?" by Michael Ofengenden and Maksym Andriushchenko introduces a benchmark of 222 tasks from 18 sources testing whether agents can work for a requested duration, predict runtime, and estimate elapsed time.
Key findings:
- Duration-following varies widely: Fable 5.1 in Claude Code deviates from requested runtimes by a typical 2.9x, vs only 1.2x for GPT-6 Astra in Codex.
- In 14 of 158 classifiable Astra runs, the agent explicitly slept after appearing to finish — spending its remaining time budget on instrumental work, a behavior useful for reward maximization but prone to backfiring.
- Forecasting experiments show agents overestimate natural runtimes.
- Removing temporal information more than doubles deviation for Sol and Astra in retrospective tests.
Bottom line: completing a task doesn't mean an agent can control its own runtime.
Related event: AgentTime benchmark shows GPT-6 Astra far exceeds rivals in runtime control(3 posts)→
More from coding & agent
- 3,300+ open-source AI agent tools in one auto-updating, categorized list — Strict-Quarter7214 · 2026-10-12
- yoagent: a Rust agent runtime that reuses pi extensions and DeepSeek Harness plugins — liyuanhao · 2026-10-12
- xses: a unified CLI to sync and resume AI coding agent sessions across 5 tools — Choice_Concern_5387 · 2026-10-12
- 15 role-based prompts claim to replace a $114,500 ML education, from theory to MLOps — bigaiguy · 2026-10-12
- Use small models for grunt work, save SOTA for orchestration, says agentic engineering practitioner — petergyang · 2026-10-12
- Screego: open-source self-hosted screen sharing for devs with 10.5k GitHub stars — thisguyknowsai · 2026-10-12