willcb: on-policy GRPO is nearly true RL, everything else is efficiency tricks
willcb · x · 2026-10-12
On whether it's right to collapse GRPO, DAPO etc. into "RL", willcb argues that on-policy GRPO (without std dev normalization) is very close to "true RL" (REINFORCE). Everything else is efficiency tricks that risk some instability, followed by further tricks to fix that instability.
Related event: willcb: RL Is Direct Objective Optimization, On-policy GRPO Comes Closest(2 posts)→
More from Research
- ThinkDrag: mmlab-ntu adds visual reasoning to drag-based image editing, claims SOTA — mmlab-ntu · 2026-10-12
- Dex-One2Many learns dexterous manipulation from one human video, +71% on unseen setups — Jusuk Lee · 2026-10-12
- Skeleton prompts in English? LASEF study finds language choice is context-dependent — HyeonSeok Lim · 2026-10-12
- DCOCO restores intra-block dependencies in speculative decoding with up to 6.6x speedup — Roy Eisenstadt · 2026-10-12
- JetBrains paper: coding-agent benchmarks should match real users' task flows — rohanpaul_ai · 2026-10-12
- Real-time chunking world model demo runs autonomously at 200ms per generation — ___Mufasaa · 2026-10-12