OSWorld-Persona benchmark: deep personalization boosts proactive-help timing macro-F1 by 2.4-3.0x

EchoShao8899 · x · 2026-11-16

Researchers built OSWorld-Persona, a benchmark for evaluating whether an agent knows when to proactively help:

Deep Personalization improves timing macro-F1 by 2.4–3.0x: Qwen-3.5-9B 0.186→0.483, Qwen-3.5-35B-A3B 0.183→0.433, Nemotron-3-Nano-Omni 0.118→0.349.

Contextual memory built by an on-device Observer further improves decisions about when to help.

Related event: OSWorld-Persona Benchmark Evaluates When AI Agents Should Proactively Help(2 posts)→

Original post →

More from coding & agent

coding & agent channel →