FlashREINFORCE: Critic-Free Single-Rollout Async RL Beats GRPO With Half the Rollouts
kastnerkyle · x · 2026-09-15
A new paper introduces FlashREINFORCE, a critic-free, one-pass asynchronous RL framework for agentic LLMs. Instead of group-based methods that repeatedly sample the same prompts, it learns from each trajectory as soon as it finishes, combining One-Batch REINFORCE, a Sequence Trust Region, and Sample-Mean Optimization.
Key results:
- Stable through 6,000 updates on DeepSeek-R1-Distill-Qwen-1.5B at 4-update policy lag
- 38.0 mean accuracy on Qwen2.5-Math-1.5B across five benchmarks, beating the reported GRPO baseline by 1.7pp with half the rollouts (256k vs 512k)
- Extends to Qwen3-30B-A3B with tool use; stable at policy lag 8
- On Qwen2.5-7B-Instruct, maintains tool calling where GRPO stops, hitting 98.3%/96.5% seen/unseen ALFWorld success
Related event: NVIDIA Open-Sources FlashREINFORCE, a Critic-Free Async RL Framework(4 posts)→
More from Research
- Heads Of Security: paper pinpoints and rewires attention heads behind prompt injections — jonasgeiping · 2026-10-31
- BATv1: 17,145 Cherimoya models map TF binding across 3,429 ENCODE ChIP-seq experiments — jmschreiber91 · 2026-10-31
- UCLA's ChairNav teaches autonomous wheelchairs to learn each rider's preferences — zhoubolei · 2026-10-31
- How much compute should data rewriting get? New scaling-law study quantifies it — ChengleiSi · 2026-10-31
- Open-source sticker library covers all 11,064 living bird species, drawn by AI — AlchainHust · 2026-10-31
- Epoch AI: AI chip compute grows 2.5x a year, down from 6x in 2023 — rohanpaul_ai · 2026-10-31