Tencent Hunyuan proposes SAT to stabilize asynchronous RL under stale rollouts
Tencent-Hunyuan · hf · 2026-07-22
Tencent Hunyuan presents SAT, a staleness-adaptive trust-region method for stabilizing asynchronous reinforcement learning.
Main contribution
- Uses detached sampled log-ratio as a practical proxy for rollout staleness.
- Identifies high-mismatch tails in each batch with staleness-based kernel scaling.
- Contracts only the sign-selected end of the PPO interval to preserve normal-token behavior while tightening risky updates.
Reported results
- Built on Qwen3-30B-A3B-Base with SGLang for inference and Megatron for training.
- On the asynchronous setup, SAT-GSPO w/ R3 reached the best observed AIME24 avg@8: 35.83 at lag 1 and 34.79 at lag 8.
- The authors argue that adaptive clipping plus routing replay jointly stabilize async RL under heterogeneous staleness.
Related event: Tencent Hunyuan Proposes SAT to Stabilize Asynchronous RL(2 posts)→
More from Research
- rauchg reproduces OpenAI's covert CPU-channel finding: chat across Firecracker microVMs with no network — ccerrato147 · 2026-10-27
- Microsoft Research: ForkPilot cuts long-horizon agent compute by up to 59% — EliasEskin · 2026-10-27
- Microsoft's TeleTune evolves agent skills from offline user telemetry — EliasEskin · 2026-10-27
- FlexGlue: sparse-to-dense matching that beats dense accuracy at a fraction of the cost — zhenjun_zhao · 2026-10-27
- PixGR: pixel-space generative decoding for fast feed-forward 3D Gaussian reconstruction — zhenjun_zhao · 2026-10-27
- Mundo turns a few room photos into an explorable, simulatable 3D world — at 1.5 FPS — mark_k · 2026-10-27