Beyond Simply Environment Scaling: Designing Effective Environment Distributions for Multimodal Agent Learning
Kejian Zhu, Zhuoran Jin, Dongqi Huang, Hongbang Yuan, Yupu Hao, Kang Liu, Jun Zhao
cs.CV
2026-08-04
Scaling multimodal training environments hurts agents. Selecting 30 by ability coverage and gradient conflict, plus a hierarchical difficulty curriculum, yields a 143.2% average gain over the base.
The prevailing recipe for training GUI and visual agents is to assemble an ever-larger pool of multimodal environments, on the assumption that more environments mean broader competence. The authors run counter-experiments that break this assumption: on Qwen3-VL-4B, scaling environments up does not lift performance monotonically but makes it oscillate, and mixing environments costs 10.7% on multimodal settings versus only 1.3% on text-symbolic ones. Multimodal agents are far more sensitive to which environments they train on, and the wrong mix is worse than no environments at all.
The authors localize the problem to two axes: whether environments overlap in the abilities they train (diversity), and whether their difficulty is sensibly ordered (difficulty structure). They turn each into a measurable quantity and address each with a dedicated method.
Two mechanisms, one for selecting environments and one for training on them.
Selection uses AES (Ability-aware Environment Selection). Each environment first gets a profile: run 40 agent trajectories (half Qwen3-VL-4B, half Gemini-3-Flash), use GPT-5 to segment them into atomic abilities, and record which abilities fire and how often; 200 environments collapse to 72 core meta-abilities. Selecting an environment then scores three things at once: how much new ability coverage it adds (NewCoverage), how much it duplicates already-selected ones (Redundancy), and whether its gradient fights the others (Conflict, measured by negative gradient cosine similarity). The score adds coverage and subtracts redundancy and conflict, and AES picks 30 of 200 while keeping full ability coverage.
Conflict is the load-bearing term. Multimodal environments show more polarized gradient similarities than text-symbolic ones, with frequent strong negative correlations, optimization directions that cancel each other. Drop the conflict term in an ablation and the relative gain collapses from 40.3% to 2.8%: the training barely moves.
Training uses HDC (Hierarchical Difficulty Curriculum), two nested levels. The outer level is harness weakening: a harness is auxiliary text given to the agent during training (textual observations, states, hints, rule descriptions), graded from fully on to fully off across H0 to H4 and removed in stages to force the model onto raw visual input. Removing it too fast breaks the model, so earlier levels are sampled with exponential decay as a buffer. The inner level is state-scale progression, controlling instance complexity per environment (grid size, say) through a sliding window that widens as the model clears a threshold. The two interlock: within a harness grade, push state-scale up until the model is competent, then weaken the harness and restart the scale.
Baselines are the raw Qwen3-VL-4B / 8B-Instruct models, all 200 environments, and 30 environments chosen at random. ST is single-turn, MT multi-turn, ID the selected training environments and OOD held-out environments.
| Model | ST (ID) | MT (ID) |
| 4B base | 13.1 | 11.9 |
| 4B + AES+HDC | 45.0 | 36.2 |
| 8B base | 17.6 | 14.8 |
| 8B + AES+HDC | 49.9 | 40.1 |
Across settings, AES+HDC averages a 143.2% relative gain over the base model. Compared with all 200 environments (43.4% relative gain) and random-30 (44.6%), the curated 30 win and still generalize to unseen environments, though the OOD absolute gains are much smaller than on ID. HDC alone lifts random-30 from 44.6% to 73.7%.
Ablations show each part carries weight: removing the conflict term drops the gain to 2.8%; harness alone gives 18.1%, scale alone 11.5%, and the two together 27.7%.
For anyone training agents, the counter-intuitive but practical takeaway is that curating the environment pool beats blindly expanding it: figure out which environments overlap in ability and fight in gradient space first. Selecting 30 beats stacking 200 while costing less compute, which matters most for compute-constrained teams. HDC's harness weakening also gives a reusable schedule, letting the model learn the action mapping with text scaffolding and then withdrawing it to force visual grounding, mirroring how humans pick up complex tasks with fading support.
This is an incremental methodological improvement rather than a new paradigm. The meta-ability profiles depend on GPT-5 annotation, and the conflict term requires offline per-environment gradients, so selection carries extra overhead.
The authors list several: the environment pool builds on existing work rather than studying large-scale synthesis; compute constraints prevented running all 200 environments under an expanded budget; AES's use of gradients adds offline computation; and more efficient conflict estimation is left open.
Reading it closely, the gains are relative to a base model that scores quite low on these agent benchmarks (4B starts in the teens), so percentage gains amplify small absolute numbers. The OOD gains are only a few absolute points, so generalization is far less dramatic than the ID numbers suggest. The 72 core meta-abilities are segmented by GPT-5, and the quality of that segmentation directly determines whether the coverage metric is trustworthy; the paper does not validate this with strict human review.