Berkeley's Exo lets agents edit their own harness mid-task, cutting model cost up to 82%

Exo: An Architecture for Self-Improving Systems at Runtime

Alexander Krentsel, Sylvia Ratnasamy, Ion Stoica, Martin Casado, Ankur Goyal

cs.AI

2026-10-31

Exo, a Berkeley-built agent runtime, lets agents edit their own harness code mid-task; across a game and three benchmarks, model cost falls 14–82% against baselines at matched accuracy.

What problem this solves

Half of an agent's capability lives in the model; the other half lives in the harness, the scaffolding around the model that assembles context, exposes tools, manages memory, and executes actions in the environment. A network-operations agent needs different tools and context policies than a coding agent, so every new domain means re-engineering the harness.

Both current paths are painful. Hand design is slow and leans on domain intuition. Automated optimizers (GEPA, Meta-Harness, AlphaEvolve) run an external loop: propose a candidate harness, execute it on a benchmark, revise. That loop assumes a benchmark exists, and a production study the paper cites found 75% of interviewed teams evaluate agents with no benchmark set at all. Task horizons keep lengthening, with reported runs lasting days to weeks, so evaluating each candidate through a full task gets prohibitively expensive. Even an optimized policy stays frozen during execution: an agent may notice that most diagnostic output is irrelevant while its context-assembly policy keeps including all of it.

Exo collapses the optimization loop into task execution: the running agent edits its own harness while it works.

Method

Two properties of harnesses make this feasible. Optimizer symmetry: revising a harness uses the same actions as ordinary work (inspecting source, editing files, running tests, requesting activation), so revision becomes just another choice available to the policy. Operational homoiconicity: the harness policy is code, stored as files the agent can already read and edit, so the object being optimized is present as data in the optimizer's environment. Together they let the optimizer move inside the harness, and since the policy governing when and how to improve is itself editable, the agent can revise how it improves itself.

Self-editing creates two systems problems, and the architecture answers them with a policy/mechanism split borrowed directly from the 1995 exokernel:

An underrated piece is internal legibility. The agent can query its full execution history with per-step costs and cache behavior. The authors argue this is central to why Exo works: efficiency signals are available mid-task, long before completion provides any reward.

Exo is open source, about 20,000 lines for the core runtime and 1,000 for the base starting policy, and has reportedly served on the order of 10k tasks and 5m messages in production.

Results

Open-world experiments run in RuneScape across three long objectives (maxing combat stats, earning currency, crafting an adamant platebody), against the Codex CLI and Pi coding agents, all on gpt-6.1-sol. Standalone starts fresh; transfer re-runs the objective on a fresh account with agent state retained:

SettingMetricExoCodexPi
Combat, standaloneCost to finish$3.71$7.26$7.95
Combat, transferCost to finish$1.92$6.87$10.89
Crafting, standaloneCost to finish$2.99$3.90$3.41
Crafting, transferCost to finish$0.88$1.89$1.61
Money-making, standalonek-coins per $332231148

That is 12–53% cheaper standalone and 45–82% cheaper on transfer, with all arms finishing the finite objectives. Learning is an up-front investment: at ten minutes Exo has spent $0.31 versus $0.12 and $0.16, with Pi's cumulative spend passing Exo's at 25 minutes and Codex's at 86. During combat Exo made sixteen self-edits, including batching melee actions into a tool that reports experience gained, skipping stalled targets, and trimming context to the last four turn summaries. The negative result is reported too: on quests, the least repetitive objective, Exo spends $0.28 per quest point versus $0.24 (Codex) and $0.21 (Pi). Self-improvement does not pay back when there is little structure to amortize.

On three graded benchmarks the control is the identical starting policy with editing disabled (gpt-5.6-sol): AppWorld for everyday app tasks, SREGym for live-service fault diagnosis, CLB for sequential issue fixing in real repos.

BenchmarkMetricControlExo
AppWorld (100 held-out tasks)Pass rate96.9%96.9%
AppWorldCost per task$0.359$0.141 (−61%)
AppWorldTotal incl. learning$57.36$32.70 (−43%)
SREGym (20 held-out incidents)Median diagnosis time81 s53 s (−35%)
SREGymMedian model rounds1610 (−38%)
CLB coding (19 issues)Resolution100%100%
CLBCost per issue incl. reflection$0.507$0.436 (−14%)

On AppWorld, cumulative cost including learning drops below control by task 9, cached input share rises from 57.5% to 86.1%, and a learned tool fetches compact docs for several APIs in one call. On SREGym, Exo builds an incident-snapshot tool, revises it five times, and opens every held-out incident with it. Weaker models break the premise: with gpt-5-mini on AppWorld, three seeds made few edits and reused none of their tools, landing at $6.63 and 59% versus $6.73 and 55%. The authors call this an intelligence floor.

Why it matters

For anyone building agents, three things carry practical weight. Harness engineering is a real cost, and this moves adaptation into the deployment environment instead of requiring a benchmark; learning happens exactly where the policy will run. The learned adaptations are explicit code diffs that can be inspected, tested, rolled back, and routed through existing code review, which makes continual learning auditable in a way weight updates are not. And the runtime is open source at roughly 20k lines, not a paper-only artifact.

Be clear about what it buys: every number is efficiency and cost. Pass rates match the control everywhere. It makes agents cheaper, faster, and portable across similar tasks; it does not make them more capable. As an incremental but solid systems contribution, that positioning is honest.

Limitations

Terms

Source

What people are saying

Related papers

All paper explainers