EVOKE swaps goals at a fixed state and ranks the same actions, reaching 92% on unseen agent tasks

EVOKE: Eliciting World Knowledge in Agents for Transferable Decision-Making

Yuhan Guo, Jinming Liu, Liang Xu, Ziqiang Li, Jianguo Huang, Zhicheng Wang, Hu Zhu, Qiuyu Chen, Yuntao Wei, Xin Jin, Wenjun Zeng

cs.CL

2026-09-30

EVOKE trains LLM agents by ranking the same candidate actions under swapped goals at a fixed state, lifting unseen ALFWorld success from 60.4% to 91.8%.

What problem this solves

LLM agents post-trained in one set of environments fall apart in unseen ones; what they learn stays tied to the training tasks. The main fix on the market is the world-model route: train the agent to predict the observation that follows each action, then plan over those predictions. That adds a prediction objective to training, much of it irrelevant to the decision at hand, and prediction errors compound when chained over multi-step planning.

The premise here is that the knowledge is already in the weights. LLM agents mostly operate in digital settings such as websites and search engines, where action consequences (placing an order ends a purchase on any shopping site) fall within knowledge absorbed during pretraining; recent work has measured that pretrained models predict text-environment transitions well from a few examples. The problem shifts from acquiring a world model to eliciting one. Standard post-training supplies no pressure to elicit: each visited state carries a single training goal, and the policy can fit the signal by memorizing the habitual next action for that context. Habits work in distribution and break outside it.

Method

The theoretical hook is Richens et al. (2025): an agent competent across a sufficiently rich set of multi-step goals must contain a world model recoverable from its goal-conditioned behavior. EVOKE turns that existence result into a training recipe. The decomposition it relies on (Eq. 1): the environment transition is identical for every goal; the goal only changes which consequences are worth having. What an action does is shared across goals; what it is worth is goal-specific.

The loop runs in four stages, iterated DAgger-style:

State collection: the policy rolls out in training environments; sampled states cover progress, detours, repeated actions, and steps heading toward failure, so supervision lands where the policy actually goes.
Goal intervention: state, visible history, and available actions are frozen while the goal is swapped for an alternative. A Qwen2.5-72B-Instruct annotator proposes candidates, each instantiated in the environment together with a success condition and kept if compatible and not yet satisfied.
Grounded assessment: candidates are really executed from the same state, with a short follow-up when one step does not reveal the effect; the annotator labels observed outcomes as positives or competitors and discards actions with insufficient evidence. It judges what happened, never what would happen.
Contrastive ranking: a listwise loss plus 0.25 times a pairwise term, with actions scored by mean token log-probability. Competitors are filled first with actions the policy itself favors but the annotator judges worse, starting with its greedy action whenever that is not a positive. Holding a dirty cup while the goal is a clean one on the table, placing the dirty cup down is the textbook hard negative.

The design logic: when the same context calls for different actions under different goals, no context-only habit can rank them correctly under both. Ranking correctly requires linking what an action does with what the current goal needs, which forces pretrained consequence knowledge into the decision. Training touches only LoRA adapters (rank 16); at deployment it is an ordinary policy with no world-model module, no inference-time planning, and no annotator.

Results

Three backbones (Qwen2.5-3B/7B-Instruct, Qwen3-1.7B) across ALFWorld, WebShop, and seven search-QA datasets: EVOKE takes the best average on every benchmark and backbone, including all world-model and dynamics baselines on 7B, and stays best on unseen ALFWorld games. Margins are largest on the multi-step environments and the smallest backbone; on search QA the baselines cluster more closely and EVOKE still leads on each backbone.

Main ablation (Qwen2.5-3B, ALFWorld, %):

Variant	Seen	Unseen	Avg steps
πboot (demonstration SFT)	69.3	60.4	25.9
Original goals only	82.9	82.8	16.0
Source replay (matched updates)	90.4	85.9	15.3
Positive SFT (same data, imitate)	91.4	84.3	15.0
EVOKE	92.1	91.8	11.4

Goal interventions carry the largest share: removing them drops unseen success from 91.8 to 82.8 and costs 40.7% more actions per game; under a fixed supervision budget, swapping half the goals gives 84.5 versus 78.8 (95% CI [1.3, 9.5]).
Ranking beats imitation: Positive SFT gets identical goal contexts, positives, and updates, nearly matches seen performance, and falls 7.5 points behind on unseen. EVOKE wins all 15 paired runs; trained on 60% of the labeled states it reaches 87.6 unseen, above full-data SFT at 84.8.
Hard negatives matter: replacing them with random valid actions drops unseen to 74.6; with the actions the policy is least likely to take, 67.2.
Data efficiency: two rounds supervise 1,482 decisions, 9.1% of the 16,247 demonstration steps behind πboot, and lift unseen success from 60.4 to 91.8; across the three backbones the two rounds add 20.1 to 31.3 points, with a clear gain in each round.
The knowledge is already there: linear probes decode five post-action consequences (held, contained, hot, clean, cool) from pre-action hidden states almost perfectly, including in the raw Qwen2.5-3B before any ALFWorld training; on transitions where identical action text yields different outcomes, representations stay above 97.0% while an action-text probe sits near chance (random-weight control 65.0 to 68.0%). Yet πboot, equally decodable, manages only 60.4% unseen. The knowledge is present; using it is a separate matter.
Behavior actually changes: on 812 goal pairs sharing one state, history, and action set, πboot solves 19.5% and more than half of its errors are habitual (it picks the action correct for the other goal); EVOKE's habitual-error rate is 6.4%, 4.3 points below Positive SFT (CI [3.0, 5.6]). It solves 91.0% of unseen games within 20 actions (at most 80.6% for the other trained variants), needs 24.4 to 28.9% fewer actions per game, commits less than half as many invalid actions as Positive SFT, and on the 70 games all five variants solve it still takes the fewest actions.

Why it matters

For agent post-training this is a far cheaper alternative to world models: no prediction objective, no extra module, zero deployment overhead. Data construction needs a resettable environment plus an LLM annotator, and executing candidates to label observed outcomes is easier to automate than hand-labeling correct actions. Two design pieces transplant independently: goal diversity as an antidote to contextual habits, and the policy's own favored mistakes as negatives. Any environment that can be simulated and whose goals can be instantiated with success conditions qualifies, which covers web, search, and tool use.

The gains come in two tiers: +31 points unseen on ALFWorld-3B with roughly half the steps is a large jump; on search QA, where baselines cluster, the margin is confirmatory. What the paper establishes is the value of this supervision design, not a solution to agent generalization.

Limitations

Label quality is capped by the 72B annotator; the paper reports no quantification of annotation error.
Every candidate action is executed and the environment reset, and goal interventions multiply state-goal combinations; the construction cost is never billed.
It requires resettable environments and goals instantiable with success conditions; live, non-rollbackable websites are out of scope as-is.
The full world-model baseline group appears only in the 7B main table (3B gets two PaW variants, 1.7B none), so the comparison against world-model methods is incompletely covered.
Physical environments are excluded by the authors themselves, since contact and motion dynamics are hard to capture without grounding; by the same logic, domains pretraining never covered (private systems, new tools) gain nothing, and that case is untested.
Iteration stops at two rounds; whether gains saturate is not measured.

Terms

world model: a model of how an environment responds to actions, used to look ahead before acting; this paper replaces prediction training with decision supervision.
DAgger: dataset-aggregation imitation learning; each round collects fresh states from the current policy and appends labels.
hard negative: a candidate the model finds attractive but that is actually wrong; pushing it below the positives is the most effective use of contrastive learning.
linear probe: freeze the model and train a linear classifier on hidden states to test whether some information is represented.
ALFWorld: a text-based household-task benchmark with seen and unseen game splits.

Source

Related papers

All paper explainers