PersonalAgentEval launches: 7 personal agents tested on real 13-month histories
HennyJieCC · x · 2026-10-15
The MatrAIx team launched PersonalAgentEval, billed as the first benchmark evaluating personal agents across diverse users, preferences, and domains.
- Agents are tested on everyday workflows (booking dinners, paying bills, managing calendars) inside accounts pre-loaded with 13 months of realistic history (347–470 records per person)
- To pass, agents must proactively surface crucial background details like unread messages, contract clauses, or promises from past chats; tasks and user simulations are human-verified
- Instinct and OpenAI Dots top the leaderboard; Instinct led proactive context retrieval, finding 64 of 72 key facts before the user mentioned them
- The site includes full leaderboards, resolution rates by capability and domain, memory, efficiency and cost breakdowns
More from coding & agent
- Where to enforce agent limits: prompt checks, scoped credentials, or an external policy layer? — Fresh_Spread_9223 · 2026-10-15
- CognitionX Emirates 2026 talk: RAG but for tools, mastering the Foundry toolbox — lee_stott · 2026-10-15
- 9 reasons your open-source PR is getting ignored, including 100% AI-generated code — jackedAJ · 2026-10-15
- OpenAI's median researcher now burns ~$600/day on internal coding agents, up ~100x in 9 months — Confident_Salt_8108 · 2026-10-15
- Claude wrote its own Direct3D 9 and x86 emulators to run kkrieger in a browser — austinc3301 · 2026-10-15
- Running Claude Code unattended for 7 hours on two YouTube channels: building automation that never fails silently — Purple_Reporter_4844 · 2026-10-15