ThunderSyncRL speeds agentic RL up to 1.9x with zero policy staleness, Stanford paper claims
heghbalz · x · 2026-10-14
- Stanford-led paper (arXiv:2610.05935, Mirhoseini group) tackles idle learner GPUs in post-training on long-horizon agentic tasks: synchronous GRPO/OPD leaves trainers idle 49.5% of step time on SWE-smith, while asynchronous training suffers policy staleness.
- ThunderSyncRL starts each trajectory's score gradient as soon as its reward is ready (GRPO) or per-turn as the teacher scores it (OPD), then takes one synchronous optimizer step — proven identical to batch-synchronous updates with zero staleness.
- Result: up to 1.9x faster to reach sync's pass@3 on SWE-bench Verified and Terminal Bench 4.0, and beats asynchronous baselines; cheaper training per performance point on 8×B200.
More from Infra
- Espresso now auto-optimizes Databricks Spark jobs and configs — mattturck · 2026-10-14
- Intel Diamond Rapids motherboard shown off: 16 DDR5 slots, MRDIMM at 12800MT/s — firstadopter · 2026-10-14
- SayGM claims 70% cheaper LLM gateway with $5M ARR and crypto referral rewards — markjeffrey · 2026-10-14
- Eaton's 800 VDC power rack draws big crowds at OCP summit — firstadopter · 2026-10-14
- Mixed-precision, mixed-accelerator samplers cut RL training cost with off-policy correction — xiuyu_l · 2026-10-14
- Pairing M5 Ultra 256GB with M3 Ultra 512GB for disaggregated prefill/decode cuts TTFT from 40s to 18s — Usual_Tackle5892 · 2026-10-14