Apple's iPrefBench tests personal agents on 130 underspecified iOS tasks: infer or ask?
hhsun1 · x · 2026-11-03
iPrefBench, built during an Apple internship, is a user-centric benchmark for personalized mobile agents: 130 underspecified tasks across 26 iOS apps grounded in a persona with 3,051 past activity events. It tests whether agents can infer preferences from history and only ask users for information they couldn't know — neither guessing nor over-quizzing.
More from coding & agent
- PSA for Vibecoders: Playwright Bundles Vulnerable Chromium, Sandbox Off by Default — Kyrannio · 2026-11-03
- Hermes adds one-command migration for Grok Bot and OpenBot users — NousResearch · 2026-11-03
- A Weekend of Nudging Got Claude to Write Workable Comedy, Plus a Better-Than-Random Discriminator — ivan_bezdomny · 2026-11-03
- Scenario open-sources 3D OS: 80 agent skills that take a sketch to game-ready 3D assets — Scobleizer · 2026-11-03
- Solo dev ships 4-player co-op game about 1287 Mongol invasion, built heavily with Claude — Ok-Pirate5732 · 2026-11-03
- DeepSeek rolls out quick custom plugin creation to bolster its AI agent push — pstAsiatech · 2026-11-03