Salesforce's StateAct Replaces Screenshots with Program State, Cutting Costs 9x
Salesforce proposes StateAct, a new framework for long-horizon computer-use agents that addresses the high cost and inefficiency of relying heavily on visual models. It achieves new SOTA on OSWorld 2.0 and significantly reduces agent operating costs, offering new insights for the field.
Confirmed
- StateAct is a code-first framework for long-horizon computer-use agents, proposed by Salesforce.
- The core idea is to use program state (files, backend, DOM) as the primary representation for agent interaction, rather than raw screen pixels.
- The paper argues that screenshots are lossy representations because different underlying states can produce the same pixel output.
- The framework calls a GUI expert model only in about 1.1% of steps where visual judgment is truly needed.
- This state-first approach reduces agent costs by approximately 9x.
- StateAct achieves new SOTA on OSWorld 2.0 and improves Opus 4.8 performance.
Why it matters
- Current computer agents typically rely on visual models to read screen pixels, incurring huge computational and financial costs. StateAct takes the opposite approach, demonstrating that interacting via program state not only drastically cuts costs but also improves performance on complex long-horizon tasks, pointing an important direction for designing next-generation efficient agents.
2026-07-28 ~ 2026-07-29 · 5 related posts
Primary sources
- [source] StateAct: Program State Cuts Agent Costs by 9x, Sets New SOTA — _akhaliq · 2026-07-28
- [source] Salesforce’s StateAct lifts Opus 4.8 on OSWorld 2.0 with state-first agents — Salesforce · 2026-07-28
- StateAct argues computer-use agents should model program state before pixels — _akhaliq · 2026-07-28
- StateAct tops OSWorld 2.0 by using program state instead of screenshots — LiJunnan0409 · 2026-07-28
1 near-duplicate retellings: LiJunnan0409