SkyRL Trains Qwen3.5-397B on Long-Horizon Agent Tasks, Pass@1 Jumps from 16% to 27%
simonguozirui · x · 2026-09-03
Mercor Research and the SkyRL team trained Qwen3.5-397B-A17B with SkyRL plus a Harbor RL recipe on APEX-Agents (long-horizon office work) off-the-shelf data, lifting Pass@1 from 16% to 27%.
The post doubles as a practical field guide for de-risking an RL run step by step: environment infra first, harness fixes and optimizations, token-in-token-out handling, an overfitting sanity run, small-model ablations, then the hero run.
Notably, gains transfer beyond the training harness: despite training on an MCP-based harness, the model improved on OpenCode and even on Terminal-Bench 2.1. Code, models, and eval traces are all open source.
Related event: SkyRL Boosts Qwen3.5-397B Agent Pass@1 from 16% to 27% via RL(3 posts)→
More from coding & agent
- Anthropic borrows lawyers' ethics for Claude agents that closed 186 deals with no human approval — rohanpaul_ai · 2026-10-18
- One prompt turns Claude Code into a full motion-graphics video pipeline with Remotion — aziz4ai · 2026-10-18
- Prompt injection via email can hijack your AI agent — jev-guard adds risk-scored guards — sujingshen · 2026-10-18
- 14-Agent AI Film Pipeline on One RTX 5090: Fixing Cut Defects Between Clips — developervkmp · 2026-10-18
- Meta paper introduces "agent plasticity": best-performing agent isn't the most efficient learner — rohanpaul_ai · 2026-10-18
- Dev ships a bushfire prediction platform in a weekend with frontier models — CtrlAltDwayne · 2026-10-18