AgentTime v1.1: GPT-6 Astra tops agent time-awareness benchmark at 51%, Claude trails
maksym_andr · x · 2026-11-03
AgentTime benchmark v1.1 combines duration following, runtime forecasting, and retrospection into one time-awareness score (200 tasks, 600 runs per agent), adding Opus 5.5, Gemini 4-Argon, GPT-6.1 Sol, MiMo v2.6, and Muse Wonder.
Key results:
- Combined: GPT 6 Astra leads at 51%, GPT 6.1 Sol 41%, GPT 5.6 Sol 27%; Claude Fable 5.1 26%, Opus 5.5 20%, MiMo Pro UltraSpeed just 6%.
- Duration following: GPT 6.1 Sol hits the requested duration within 5% on 65% of runs (1.15× error); Opus 5.5 only 6%, Muse Wonder misses by 9× typically.
- Forecasting: poor across the board — best is Opus 5.5 at 2.26× typical miss; most agents systematically over- or under-estimate.
- Retrospection: GPT 6 Astra best (1.10× native error); errors grow for all once time cues are stripped.
Takeaway: GPT agents are clearly the most time-aware; Anthropic and others lag significantly.
More from coding & agent
- Model swap cut costs 59% and latency 75% — and he still blocked the release — Fun_Employment6042 · 2026-11-03
- Context drift broke a production LLM deployment by turn 10 — no jailbreak needed — GasProfessional5544 · 2026-11-03
- Obsidian dev open-sources Pele, a Mermaid-compatible renderer 100-500x faster than mermaid.js — Teknium · 2026-11-03
- Agent-built eval harness spawns subagents via codemode in one durable session — mitsuhiko · 2026-11-03
- SurfSense runs Claude Code and opencode as dual engines — lessons from a local NotebookLM — FurtiveMirth · 2026-11-03
- Impeccable crosses 100k GitHub stars, with weeks of releases teased ahead — jamdac · 2026-11-03