Paper: SAO explores async optimization for long-horizon agents
khademinori · x · 2026-08-21
This paper proposes "Single-rollout Asynchronous Optimization (SAO)" to address stability and off-policy challenges in asynchronous RL for long-horizon agentic tasks.
Key Insight: The next scaling law may focus less on parameters and more on how long an agent can work, the amount of real-world feedback it absorbs, and learning speed. SAO is a step toward the future of Long-horizon Agents → Continuous Learning → AI Workforce.
Technical Details:
- Context: Existing RL pipelines for LLMs are mostly synchronous and batch-interleaved, inefficient for long-horizon tasks.
- Method: SAO replaces group-wise sampling with single-rollout sampling to reduce off-policy effects and improve generalization. It introduces a strict double-side token-level clipping strategy for optimization stability.
- Results: SAO trains stably for one thousand steps and consistently outperforms GRPO and its variants on agentic coding and reasoning benchmarks like SWE-Bench Verified, BeyondAIME, and IMOAnswerBench.
Related event: Zhipu's SAO: Stable Asynchronous RL for Long-Horizon Agents(2 posts)→
More from coding & agent
- Every publishes 8,000-word Codex oral history, agent-native so your AI can dig the raw interviews — danshipper · 2026-10-16
- Developer rebuilds GitHub in 4 days: Keel makes agents declare intent before changing code — threepointone · 2026-10-16
- Stop using small models as judges with overloaded context, dev warns — xeophon · 2026-10-16
- Meta's open-weight Muse Spark Open hits Ollama with 1M-token context and Claude Code support — ollama · 2026-10-16
- Monid launches: one balance, pay-per-call access to 1,700+ tools for AI agents — HeyAmit_ · 2026-10-16
- OpenAI publishes Sites tutorial: build and share your first Site — OpenAIDevs · 2026-10-16