SkyRL field guide: RL training lifts Qwen3.5 agent Pass@1 from 16% to 27%
SinclairWang1 · x · 2026-09-03
charlieruan worked with Mercor Research and the SkyRL team to train Qwen3.5-397B-A17B on APEX-Agents (long-horizon office work) using off-the-shelf data with SkyRL, improving Pass@1 from 16% to 27%.
The post is a practical field guide for de-risking an RL dataset step by step:
- Environment infrastructure
- Harness fixes and optimizations
- Token-in-token-out validation
- An overfitting run
- Small-model ablations
- Then the hero run
Notably, gains transferred to OpenCode despite training on an MCP-based harness, and even to Terminal-Bench 2.1. Code, models, and eval traces are all open source.
Related event: SkyRL Boosts Qwen3.5-397B Agent Pass@1 from 16% to 27% via RL(3 posts)→
More from coding & agent
- Anthropic borrows lawyers' ethics for Claude agents that closed 186 deals with no human approval — rohanpaul_ai · 2026-10-18
- One prompt turns Claude Code into a full motion-graphics video pipeline with Remotion — aziz4ai · 2026-10-18
- Prompt injection via email can hijack your AI agent — jev-guard adds risk-scored guards — sujingshen · 2026-10-18
- 14-Agent AI Film Pipeline on One RTX 5090: Fixing Cut Defects Between Clips — developervkmp · 2026-10-18
- Meta paper introduces "agent plasticity": best-performing agent isn't the most efficient learner — rohanpaul_ai · 2026-10-18
- Dev ships a bushfire prediction platform in a weekend with frontier models — CtrlAltDwayne · 2026-10-18