HF engineer shares her harness for pseudo-annotating CUA agent tasks on the live web
mervenoyann · x · 2026-09-20
Hugging Face engineer Merve shares a side project: pseudo-annotating datasets for CUA agents that operate on the live web. Key challenges:
- Web datasets go stale as sites constantly change, so you must pseudo-annotate per problem and define verifiable/unverifiable completion criteria.
- Anti-bot walls are a hard blocker: her lean harness with openenv + helium on an HF sandbox couldn't bypass Cloudflare (browserbase handles this well).
- Rewards: grade unverifiable parts with a large VLM, compute verifiable rewards from DOM signals.
She admits there's plenty of know-how she lacks, which makes it fun.
More from coding & agent
- Scoble says an AI agent wrote every word of an entire book — Scobleizer · 2026-09-20
- "Look at your data": dev mocks reflex to spin up another agent — chrisalbon · 2026-09-20
- Evaluating 7 Models Across Claude Code, Codex, and Pi: Harness Choice Drives Cost, Not Success — CShorten30 · 2026-09-20
- Open-source Flywheel gives AI outputs offline re-verifiable proof receipts — MeAndClaudeMakeHeat · 2026-09-20
- Dev builds MCP middleware that scrubs personal data before it reaches the AI's context — Danielloesoe · 2026-09-20
- If early autocomplete-era LLMs could code, do we even need post-training? — menhguin · 2026-09-20