Five user roles set agent autonomy while model capability stays fixed

Levels of Autonomy for AI Agents

K. J. Kevin Feng, David W. McDonald, Amy X. Zhang

cs.HC, cs.AI

2025-06-14

UW defines five autonomy levels by user role, operator through observer, plus certificates and assisted evals that separate autonomy from capability.

What problem this solves

Agent products treat fewer questions to the user as the upgrade. Demos that fix GitHub issues, shop, or click through a screen, and the capability benchmarks beside them, all score whether the agent finishes alone. Anthropic's Responsible Scaling Policy places low-level autonomous capabilities in a higher risk class. Some researchers argue that fully autonomous agents should not be built. Capability keeps rising. How far the user stays in the loop is still not a separate control.

This University of Washington essay isolates that control. Hold the model, the tools, and the environment fixed, and a developer can still decide how deep the user's involvement goes. Autonomy means how far the agent is designed to run without that involvement. The user may be a human, or another agent that sent the original request.

Method

Levels follow the user's role. One computer-using agent walks through all five so capability stays put: a frontier model on par with OpenAI's CUA, able to click a UI, browse, run code, and write documents, sharing one computer with the user. Every level gets the same request, to explain the US economic impact of generative AI after ChatGPT's 2022 release.

Agency and autonomy are different levers. Agency is the capacity to form an intention and carry it out, so a small tool set means low agency. Autonomy is how long the design allows the agent to continue without asking. A calculator running unattended is low agency and high autonomy. A tool-rich agent that checks every step is high agency and low autonomy. Strong capability is useful at all five levels.

An autonomy certificate is issued by a third party for a stated model, prompt, tool set, and environment, and it names a maximum level. The essay points to the UK AI Security Institute, METR, and commercial bodies. The developer submits an agent, deployed privately for testing, and an autonomy case. The case follows the shape of a safety case. The claim to prove is that behavior will not go above one level. A private evaluation has to match before the certificate is issued. A new tool that can edit a database, or a move from a document editor to a collaborative whiteboard, kills the certificate.

Assisted evaluation asks which grade of user involvement is the least that still beats a threshold T on accuracy or pass rate. OSWorld, TheAgentCompany, and DiscoveryWorld contain 369, 175, and 120 tasks, and they measure completion only. Reading the implementation is a cheap screen and misses the handoff. A standby user waits beside the run: a person in a user study, or another agent in simulation. The first round has no help. If every main task clears T, the label is L5. If not, add approvals and rejections, then consultation, until the run passes. Logs are then matched to the framework's characteristics. Labeling an agent L1 from zero involvement takes five rounds. A team can also start at a guessed level and add or remove involvement from there.

Results

There is no experiment. There is no completion-rate comparison against any system, and no measured T from an assisted evaluation.

ComparisonWhat the essay concludes
Completion benchmarksOSWorld (369), TheAgentCompany (175), and DiscoveryWorld (120) record whether the task was finished. They do not record a level of autonomy.
Anthropic RSP and DeepMind's Frontier Safety FrameworkRisk is tripped by finishing multi-step tasks alone. An L4 agent that can earn revenue after one approval is close in risk to an L5 agent that earns it alone, yet a pass/fail test still sorts them into different safety tiers.
All L1 or all L5L1 agents wait on an operator to assign work. L5 agents rarely talk to each other, and the system is hard to steer, debug, and audit.

No figure shows that a higher level performs better. With nobody correcting the run, small errors compound across steps. User involvement is treated as possibly worse in two situations only: the material is beyond the user's grasp, or the agent is locked in a sandbox.

Why it matters

The same CUA-class model can be a Copilot-style L1 or an L5 system whose only control is an emergency stop. The difference is where control sits.

A certificate stored in metadata lets other agents read the maximum level. A fleet of L1 agents idles. A fleet of L5 agents is hard to debug. Mixing levels, or running a group of L2 collaborators, is easier to steer. Safety checks can split the same way: long-horizon planning for L4 and L5, and the timing of user feedback for L2 and L3.

The piece is the PDF companion to a Knight First Amendment Institute web publication. The five levels, the certificates, and assisted evaluation are proposals. A team can take the role language now. How to stop approval from turning into a rubber stamp, and how a misaligned agent can spend user inattention to gain more room, are still open.

Limitations

Each level is left as open questions, and the running example is descriptive. The essay itself puts the cost of an L1 label at five evaluation rounds.

No agent is actually graded, no certificate is issued, and T is never measured on those three benchmarks. Nothing specifies how logs map onto the characteristics column, or whether two raters would agree.

A changed tool or environment voids the certificate, so keeping autonomy independent of capability is hard to maintain once a system is deployed. There is no template for an autonomy case and no adversarial test. An agent that describes a consequential action as routine may get past a standby approver. The L4 section already says that reliably telling which actions are consequential is difficult.

The user is whoever sent the first request. A later clarifying question does not flip that identity. When agents delegate to each other, the user role changes sides, and a single maximum level does not cover the relationship.

Terms

Source

What people are saying

Related papers

All paper explainers