How I AI agent bench: Grok Bot wins big at 93%, Claude Duet trails at 79%
lennysan · x · 2026-10-29
Lenny's Newsletter amplifies Claire Vo's 'How I AI Personal Agent Bench', which tested 4 personal agents on 9 tasks.
- Grok Bot (93%, 148/160) won by a large margin, taking tasks like inbox-to-to-do and a Reddit Tokeno investigation, with the top quality score of 32
- Muse placed second (89%, 107/120): its two misses (Slack reply, bug fix) required unconnected Slack/GitHub and counted as N/A; on the 6 tasks it ran it averaged 4.0/5 quality and was fastest on 4
- ChatGPT Dot came third (88%, 141/160), winning the 'reply as me' task
- Claude Duet trailed (79%, 126/160), struggling on the collaborative bug-fix task
The full episode is on YouTube.
More from coding & agent
- Grok voice agent breaks down, sparking rant against renting other people's agent harnesses — alexcovo_eth · 2026-10-29
- A silent skip counted as success: rebuilding a bot lane that can't go quiet unnoticed — petrusenko_max · 2026-10-29
- Independent dev: 'I just talk to Claude and the feature gets implemented' — evielync · 2026-10-29
- Dev argues subagent fleets are pointless with Claude Opus 5.5 — one-on-one prompting wins — ryunuck · 2026-10-29
- The "Brain Sandwich": a three-step workflow for using AI without atrophying your engineering judgment — bibryam · 2026-10-29
- Nolan Lawson benchmarks the AI cost of JS frameworks: results mostly similar — hichaelmart · 2026-10-29