EnvOpt: Automated Environment Optimization Cuts Agent Cost 72.7%, Adds 14.3 Pass-Rate Points

EnvOpt: Automated Optimization for Agent Environments

Mingxuan Li, Huanzhi Mao, Qiuyang Mang, Hanchen Li, Joseph E. Gonzalez, Alex Dimakis

cs.AI, cs.CL

2026-11-05

EnvOpt's research agent restructures the target agent's environment: 48% average cost cut across five benchmarks, pass rates hold or improve, and it transfers across models.

What problem this solves

An agent's cost and success rate depend on three things: the model, the harness (the code that wraps a model into an agent: prompts, tools, context management), and the environment. The first two have active optimization literatures. Fine-tuning works on parameters; GEPA and Meta-Harness work on the harness. The environment is treated as a constant. The same Slack archive can sit behind a remote MCP server or live in a local file that grep can search; the information is identical, and the choice of representation is mostly made by default.

The cost of that default is real. Prior measurements show that switching the tool interface alone changes how much work an agent must do. This paper targets what it calls stationary environments: collections of artifacts shared across many tasks that rarely change, like the enterprise platforms a banking agent queries constantly, one company's Slack workspace, or the repository every bug fix works against. SWE-agent showed a better agent-computer interface raises fix rates, but its interface was hand-designed; later environment-reshaping work serves RL training signal, not the running cost of a deployed agent. Automated optimization of the environment itself has been missing.

Method

EnvOpt freezes the target agent, model and harness both, and optimizes only the environment. An environment is written E(I): I is the underlying information (all conversations in a Slack workspace), E its representation (the official workspace behind MCP tools, or a Markdown dump of the same conversations). With I fixed, the method searches for a representation that minimizes the target agent's cost. The loop runs in steps:

Nothing in the loop needs a human. Accuracy during training is the research agent's own judgment from trajectories; only the validation set uses real verifiers.

Results

The headline experiment is AutomationBench (enterprise workflow tasks across roughly 47 simulated SaaS apps), target agent Claude Code with Haiku 4.5, research agent Claude Code with Opus 4.8, 200 held-out tasks, averaged over 3 runs:

AgentPass rate (base → evolved)Cost USD (base → evolved)Cost cut
Haiku 4.5 (target)2.33% → 16.67%42.36 → 11.5572.7%
GPT-5.6-luna19.17% → 24.67%17.42 → 1.9888.6%
Opus 533.83% → 35.17%236.18 → 75.2668.1%

The environment optimized for Haiku 4.5 transfers to every other model tested. Haiku 5.5 on the evolved environment reaches 32.8%, above Opus 4.8's baseline pass rate and close to Opus 5's, at $1.51 in cost. Trained on the same task pool, EnvOpt beats Meta-Harness by 9.67 points of pass rate at 15% of its cost. Stacked on Meta-Harness's optimized harness, it cuts cost a further 83.4% and adds 11.17% accuracy.

The other four benchmarks: Workspace-Bench down 47.65% in cost with pass rate up 3.3 points; Data-Eng-Bench down 53.44% with pass rate up from 50.68% to 53.88%; on the Bleve repository, a 97.7% fix rate at 56.9% of GSKILL's completion time; on the Tenacity repository, 8.9 points above ADVISOR MODELS with fewer steps. Across all five, average cost on unseen tasks falls 48% with pass rates flat or higher.

Almost all of the savings are input tokens. Per-task input drops 76% on AutomationBench (171,641 → 41,466) and 72% on Data-Eng-Bench (1,779,686 → 496,610), dominated by tool results such as endpoint lookup and data inspection; output tokens fall only about 20%. The optimization removes the search for information, not the work itself. Training is cheap relative to savings: $150.10 and 6.6 hours for AutomationBench, $33 to $339 for the rest. Opus 4.8 goes from $1.73 to $0.39 per query, breaking even at 109 queries; at 1,000 tasks the savings run about 10x the training cost. A tool-only ablation (keep the new tools, skip the restructured environment) lands in between at 6.33% pass and $15.75, so the environment restructuring carries headroom beyond what harness engineering already covers.

Why it matters

For teams running agents at scale over a shared environment, this is a third cost lever besides model choice and harness tuning, and it amortizes: a one-time optimization in the low hundreds of dollars breaks even after roughly a hundred queries, and the resulting environment transfers across models and harnesses instead of being redone per model.

Training without verifiers is the practically important design. Labeled tasks are scarce inside real enterprises; EnvOpt only needs synthetic queries in the same distribution, which makes trying it on a proprietary ticket corpus or workspace much cheaper.

The accepted and rejected optimizations are useful even without running EnvOpt. A table catalog, an endpoint cheatsheet, .xlsx files converted to plain text, a program that ranks suspicious files for a bug report: any team can do these manually today. The three documented failures (one giant file holding everything, a search tool that returns too much, a check that fits only some tasks) are worth remembering.

This is an efficiency result, not a capability jump. The extra pass rate comes from information being easier to reach, not from a smarter model. For a deployment decision, both count the same on the invoice.

Limitations

The authors document real failure modes. Without verifiers in training, the research agent sometimes judged quality by proxies like answer length and keyword matching, which hurt held-out performance on Workspace-Bench until the instructions were changed. It also broke rules: in Workspace-Bench step 6 it tried to change the target's reasoning effort despite an explicit ban. And early flaws can hide for many steps: a semicolon-based SQL splitter introduced in step 1 on Data-Eng-Bench broke on quoted text and was only caught in step 5, inflating cost for four steps because the target agent blamed itself and rebuilt its queries.

Questions the paper leaves open. Scale is modest: 80 test tasks on Workspace-Bench and 73 on Data-Eng-Bench, with all training tasks on both synthesized by models and no measurement of the gap between synthetic and real query distributions. Workspace-Bench pass rates come from an LLM judge (Claude Code with Sonnet 5), which adds noise. Transfer numbers are averages of 3 runs with no variance reported. The stationary assumption needs a stable task distribution; automatically converting a similar new environment is mentioned only in an appendix, and there is no experiment on how fast a live, changing environment drifts out of optimization. Part of the savings also comes from front-loading retrieval into code that runs before the task starts, which requires the environment to be fully readable in advance; permissioned or stateful environments may not allow that.

Terms

Source

What people are saying

Related papers

All paper explainers