SAO Outperforms GRPO Variants
ziv_ravid · x · 2026-07-11
This thread discusses a new RL training approach and shares experimental results:
- Is SAO slower/more stable to converge than GRPO? The original text clarifies that the authors found GRPO degrades faster, while SAO does not, and that SAO outperforms GRPO variants on SWE-Bench Verified and math benchmarks.
- They also conducted an experiment: when reward preferences changed midway through training, the value model tracked the new preferences faster than a "running-mean baseline."
This post is essentially a sharing of research observations regarding agent/RL training stability and the role of value models.
Related event: GLM Team Proposes SAO Algorithm for Asynchronous Agent RL(15 posts)→
More from Research
- AI papers may be easy to generate, but most still look trivial without human input — _akpiper · 2026-07-21
- CPC-Bench uses 7,102 NEJM clinicopathological cases to test medical diagnosis — GlassHealthHQ · 2026-07-21
- CPC-Bench adds 7,102 physician-validated NEJM cases across 1923–2025 — GlassHealthHQ · 2026-07-21
- A math researcher’s AI workflow review ranks models on differential geometry — BLUECOW009 · 2026-07-21
- Gritt raises a new round to automate solar array installation and maintenance — rebeccakaden · 2026-07-21
- NSA's Mike O'Hara: AI Puts Math Research Progress on 'Fruit Fly Years' — AlexKontorovich · 2026-07-21