SkillRise: Agentic Reinforcement Learning for Cross-Task Skill Evolution
Zhiyuan Yao, Yuxin Chen, Zhengxi Lu, Zishan Xu, Yueqing Sun, Yifu Guo, Yuquan Lu, Zhengzhou Cai, Kangning Zhang, Zhuowen Han, Zi-Han Wang, Ziang Ye, Qi Gu, Xunliang Cai, Weiwen Liu, Yongliang Shen
cs.LG, cs.AI
2026-07-29
SkillRise orders related tasks easy-to-hard and has one policy alternate solving and rewriting a shared skill document, leading Pass@1 by 2.3 to 8.5 points over the strongest baseline.
Agents often meet tasks that are related but not identical and that share solution patterns: different household chores in ALFWorld, different science experiments in ScienceWorld. Standard agentic RL treats each task as an independent episode with nothing passed between them. Existing skill-learning methods either hammer one task repeatedly or run a multi-stage pipeline that entangles skill extraction, retrieval, and execution, which is heavy and lets the stages interfere. The common thread is that cross-task experience never gets explicitly consolidated.
SkillRise orders related task instances, by environment-provided family metadata, from easy to hard, then has one policy alternate between two roles. The solver role completes the current task; the curator role rewrites a skill document that is passed straight to the next task in the sequence, serving as the sole information channel between tasks.
Curation isn't a running log. The policy rewrites the whole document, keeping useful skills, consolidating successful procedures and typical failure modes, and dropping instance-specific detail. The key design is decoupled credit assignment: the solving step is scored only on whether the current task succeeded, while the curation step is scored on discounted cumulative outcomes of downstream tasks, with a cross-task discount gamma letting future reward backprop into the current rewrite. Optimization is role-aware group-relative, computing advantages only across trials that share the same task family, sequence stage, and behavioral phase.
Pass@1 for Qwen3-4B on three text-based agent benchmarks:
| Benchmark | SkillRise | vs. strongest baseline GiGPO |
| ALFWorld | 85.9 | +2.3 |
| WebShop | 84.4 | +7.1 |
| ScienceWorld | 54.6 | +8.5 |
On Pass@3, ScienceWorld reaches 61.0, ahead of LaMer (which trains on repeated attempts of one task) by 14.2 points. Two scaling patterns stand out. Cross-task test-time scaling is the standout: holding out the same 128 tasks and partitioning them into related sequences of length K, attempted once each, success rises from 83.6 (K=2) to 87.5 (K=6), and the gap over single-task baselines widens from 3.9 to 8.6 points, showing the gain is cross-task skill reuse, not same-task oversampling. Model scaling from 1.7B to 4B: SkillRise improves 7.8, versus 4.7 for GRPO and 3.3 for LaMer. In engineering terms it runs 6.0x faster than RetroAgent and 4.3x faster than SkillRL.
For agent-training practitioners, this compresses learning skills from a standalone multi-stage pipeline into a single role inside one policy: lighter to build, and reusable across tasks. Cross-task test-time scaling deserves particular attention: the more tasks and the more related they are, the bigger the payoff, opening a new growth dimension for agents in long-chain workflows.
The authors concede three boundaries. It needs environment-provided task-family metadata to build the sequences, which real businesses may not have in clean form. Experiments stop at 4B parameters. It covers only three text-based benchmarks with verifiable outcome rewards; continuous action spaces or fuzzy feedback aren't tested. One concern reading through: all three benchmarks are fairly structured with clear success criteria, where an explicit memory like a skill document naturally fits. Whether this transfers to open-domain dialogue or long-horizon real tasks, where it's unclear what to write and how long, the paper offers no evidence.