Deep personalization boosts on-device agent timing macro-F1 by 2.4–3.0x

EchoShao8899 · x · 2026-11-17

On OSWorld-Persona, the author shows Deep Personalization improves when-to-help timing macro-F1 by 2.4–3.0×: Qwen-3.5-9B goes 0.186 → 0.483, Qwen-3.5-35B-A3B 0.183 → 0.433, and Nemotron-3-Nano-Omni 0.118 → 0.349. Across 10 real users with varied AI familiarity, Coco's macro-F1 proxy rose from 0.199 to 0.515 in 7 days, as learned preferences evolved from broad cues to workflow-specific decisions. Contextual memory built by the on-device Observer also improves timing judgments.

Related event: Coco: An On-Device Desktop Agent That Learns When to Help(9 posts)→

Original post →

More from coding & agent

coding & agent channel →