Agent Benchmark Addendum: Runs Take 37-85 Minutes, Scores Fluctuate
PawelHuryn · x · 2026-10-14
Pawel Huryn shared additional data from his AI agent benchmark: the three isolated runs took 37.4, 47.3, and 84.7 minutes, scoring 40, 45, and 37 respectively. Combined with the previously published costs ($5.74/$6.43/$9.30), the numbers highlight significant variance in cost, time, and quality across isolated runs of the same task.
Related event: AI Agent Benchmark Costs and Variability Revealed(2 posts)→
More from coding & agent
- Meta launches coding agent Muse Wonder Full at $50 or $200 per month — lemire · 2026-10-14
- Agent Baseline: Docker, Barclays et al. Publish Open Reference Architecture for Securing Autonomous Software Factories — AccBalanced · 2026-10-14
- Agent spots 54 fake-looking details in a Claude-built skateboarding game over five iterations — prasenx · 2026-10-14
- He runs 5 one-person companies in Cursor toward $50K MRR with this OS — PrajwalTomar_ · 2026-10-14
- Meta paper: reward hacking spreads between agents in multi-agent coding systems — omarsar0 · 2026-10-14
- One Prompt, an Infinite 3D Musical Garden: 82,000 Grass Tufts and a Gesture-Played Harp — AI_Andrew · 2026-10-14