StatePortal synthesizes agent checkpoints for GUI deep-state testing, 0%→27.5% completion
SJUT1 · hf · 2026-11-19
A Shanghai Jiao Tong University-led team introduces StatePortal, which tackles the 'deep state' bottleneck in GUI software testing — states requiring long interaction histories whose cost recurs across commits. An agent synthesizes checkpoints from natural-language test cases plus read-only source access, restored via per-app hooks bound or synthesized automatically without hand-written code.
- Hybrid programmatic-state + visual oracle scoring; visual oracle calibrated against a blinded cross-VLM reference
- DeepStateBench released: 61 deep-state tasks across 10 apps, 5 language families, 6 rendering stacks, 874 lines to 7.2M LOC
- Under the same pixel action space: baseline GUI agent 0%, checkpoint-seeded agent 10.4%, StatePortal 27.5%
- 98.90% of 1,361 replays across 112 checkpoints reproduce verdicts with zero model calls; the hybrid oracle catches 96.25% of visual faults invisible to state-only checks
Code and website are open-sourced.
More from coding & agent
- Microsoft Decision 1.1 demo: photo-to-Minecraft-skin in one parallel model call — jennifermarsman · 2026-11-19
- Claude autonomously made an 'unnecessarily detailed' Domino's pizza video — full prompt inside — ABSOLUTE-DEVELOPER · 2026-11-19
- 'Evals for AI Engineers' print copies arrive, same day as GTA VI — HamelHusain · 2026-11-19
- Netlify launches Server: always-on backends for realtime apps and AI agents — thisiskp_ · 2026-11-19
- OpenAI researchers value AI coding agents more than half their colleagues as token costs top salaries — The Decoder · 2026-11-19
- Building the Agent Took an Afternoon; the Data Integrations Ate the Whole Week — ConfectionNo6117 · 2026-11-19