2026-08-28
Banyan varies layouts, objects, and task-tree topologies independently: diversity drives transfer gaps to ~0, but n=256 plateaus near 0.80 success versus 0.95 at n=16.
Diverse training tasks improve zero-shot generalization after weights are frozen. That pattern is already visible in large domains such as XLand and AdA. Continual RL asks a different question: the task distribution keeps changing and weights keep updating, so does diversity make the learning process itself more transferable? Existing benchmarks either stop at a few hundred tasks or generate huge procedural sets without a way to turn individual diversity axes on and off.
Banyan, from Harvard / Kempner, isolates the question. In one GPU-accelerated MiniGrid-style domain, layouts, task-tree instances, and task-tree topologies can be swept independently.
A task is a triple of layout L, topology T, and object assignment O. The layout sets walls and initial positions. The task tree says which leaf objects must be transformed or pairwise combined to produce the root goal. The agent sees the full grid and a description of the goal object, not the tree. Actions cover movement, pickup, drop, and single-object transforms. Producing the root yields +1; a merge that makes the instance unsolvable yields −1. Depths 1 through 6 nest, so deeper tasks contain shallower ones, and diversity is counted by the number of depth-6 tasks.
Forward transfer is the gap Δi between terminal success on one distribution and initial success on the next; near zero means the agent barely drops at the boundary. Backward transfer is how later training changes earlier distributions; negative values are forgetting. Algorithms are kept minimal: PPO and PQN, plus Continual Backprop on the ten-shift sequence as a check against loss of plasticity.
Single-shift protocol: vary one axis in d1, then move to 10k held-out layouts or object assignments, or 256 held-out topologies, in d2. The ten-shift protocol co-varies layouts and topologies from n=1 to 256, up to 65,536 unique tasks per distribution. PPO uses 5 seeds, PQN 3.
On a single shift, raising diversity along any axis drives Δ2 toward 0: the agent enters d2 near the success rate it had just reached on d1, for both PPO and PQN. Backward transfer improves clearly only when topology count varies; layout count and object-assignment count barely move that metric. PQN's TD-error at the boundary falls as diversity rises, so features predict off-distribution returns more accurately. A continuous-control Point Mass replica shows the same direction: raising object count from 1 to 1,000 drops Δ2 from 0.95 to 0.50 and B(2,1) from about 0.80 to about 0.01.
A ten-distribution sequence reverses the long-run picture. Diversity still flattens every boundary Δi, yet later learning stalls.
| Diversity n per phase | Terminal success on d1 | Terminal success on d10 | Boundary drop |
| 16 | 0.74 | 0.95 | 0, then keeps climbing |
| 256 | 0.80 | 0.80 | 0, then flat |
End-of-d10 success falls monotonically as n grows past 16: 0.95, 0.91, 0.86, 0.82, 0.80. Backward transfer moves the other way. At n=256, success on d1 rises from about 0.80 at the end of phase 1 to about 0.95 after phase 10. The network keeps fitting structure shared across distributions and stops specializing to what each new one adds. Continual Backprop does not lift the plateau.
"Add more diverse tasks" helps frozen-policy zero-shot evaluation and can hurt continual RL that still updates weights. Diversity buys a soft landing at each shift and spends the ability to specialize afterward. Moderate diversity (n=16 here) is the setting where the ten-phase sequence actually keeps improving. Anyone scaling multi-task or continual RL should not treat task count as a dial that only goes up. Banyan splits layout, instance, and topology, which makes that trade-off measurable on an academic budget.
The main results are grid-world. Continuous control only repeats the single-shift sweep, with no ten-shift sequence and no visual diversity. Agents are recurrent throughout; whether transformers stall the same way under high diversity is untested. Baselines are PPO, PQN, and CBP. Replay, modular policies, or model-based planning might recover the compounding benefits that vanish here, and the paper does not check. The authors favor interference over failed optimization because CBP does not help and old-task performance keeps rising. That still leaves representation collapse versus capacity contention unseparated.