Annotating CUA agent data: tasks go stale, so pseudo-annotate the tasks themselves
mervenoyann · x · 2026-09-20
Hugging Face engineer Merve Noyan shares a side-project lesson on building datasets for computer-using agents: most web task datasets are stale because websites change constantly. Her key insight is to pseudo-annotate the problem/task itself from scratch — not just answers to a given problem — since tasks also go outdated, then have agents visit real sites, execute the task, and define completion criteria for rewarding.
Related event: HF Engineer Shares Lessons on Pseudo-Labeling Data for Web Agents(2 posts)→
More from coding & agent
- Dev picks C for new engine: not for AI, but full control and universal bindings — gdechichi · 2026-09-20
- Dev take: AI's million-lines-a-day code is slop, public perception lags — BLUECOW009 · 2026-09-20
- chatpipe-mcp Lets AI Coding Agents Publish Live Pages With Shareable URLs — modelcontextprotocol · 2026-09-20
- Agentic Web talk: generations already replace Chrome with AI assistants — EdenEmarco177 · 2026-09-20
- A Mac drawing app added an MCP server to test its tools—and it became the killer feature — DrawSimple_for_MacOS · 2026-09-20
- Hugo Bowne offers free 30-minute Lightning Lesson on AI agent evals — hugobowne · 2026-09-20