Why batch size decides PPO vs GRPO: a directional bias-variance view
stochasticchasm · x · 2026-11-11
A research thread introducing a directional bias-variance tradeoff view of policy gradient advantage estimators. Each estimator is characterized by alignment c (cosine similarity between its expected gradient and the true policy gradient) and noise scale ν (single-trajectory gradient variance relative to the squared norm of the expected gradient), yielding E[cos(ĝB, g)] ≈ c/√(1+ν/B). Since PPO typically has lower alignment and noise scale than GRPO, this explains why PPO wins at small batch sizes while GRPO wins at large ones.
Related event: Researchers Frame PPO vs GRPO as Bias-Variance Trade-off(2 posts)→
More from Research
- AI×Antibodies hackathon in Oslo offers JapanFold, Anthropic credits, $20K lab validation — DavidBennett__ · 2026-11-11
- Which benchmarks do humans still win against frontier AI? Reddit debates AGI criteria — ImproveOurWorld · 2026-11-11
- EarthRecord benchmark tests 12 multimodal agents on four decades of raw satellite imagery — AIEyesTech · 2026-11-11
- WaveGen: codec-free end-to-end diffusion hits SOTA text-to-speech via Spectral Forcing — Ajou · 2026-11-11
- Berkeley's Social Humanoid teaches Unitree G1 socially responsive interaction — berkeley_ai · 2026-11-11
- Robotics is not solved: frontier models are ~6 orders of magnitude off real assembly lines — ZeeshanZiaML · 2026-11-11