Google & HKU's VeriCUA cuts false task-completion rate to 5.4% by checking saved files, not agent claims
rohanpaul_ai · x · 2026-11-03
Google and the University of Hong Kong released VeriCUA (arXiv 2610.40578), arguing an agent's own "task done" message should never count as proof — saved files and app state should be checked instead.
- Common verifiers judge a few screenshots (VLM judge) or run one generated script; both miss decisive evidence
- VeriCUA explores the environment to build a requirement checklist, then runs GUI or code checks (e.g. file reads) until each item is settled
- Results: 5.4% false-positive rate on ScienceBoard vs 24.8% for a VLM judge and 21.7% for a script verifier; rejected 80% of false completion reports across 3 online environments
If agent runs feed evals, training data, or RL rewards, verify saved machine state rather than trusting screenshots or the agent's report.
Related event: Google and HKU Propose VeriCUA to Catch False Task Completion by AI Agents(2 posts)→
More from coding & agent
- DeepSeek publishes prompts for all 7 Harness Creator Mode plugin demos — aigclink · 2026-11-03
- DeepSeek Harness's Creator Mode generates plugins from plain language, threatening micro-SaaS tools — aigclink · 2026-11-03
- A one-night AI-built project nets 333 points and ~22k visitors — Aizkmusic · 2026-11-03
- Five manual checks before trusting an agent actually finished its job — Kodenstudio · 2026-11-03
- Open-source AI pit crew for iRacing runs 5 agents on Kimi, checks every number against telemetry — JeremyCMorgan · 2026-11-03
- Agent control plane benchmark: Smalltalk-80 cascades hit 100% compliance vs 2.5% for Gherkin — RandalSchwartz · 2026-11-03