PufferLib author: 420-step, 4k-rollout config trains Breakout in 1 second on 1 GPU
jsuarez · x · 2026-09-22
Joining the RL config discussion, the PufferLib author shared their setup for comparison: 420 steps, 4k rollout segments (32 horizon), async 1 — enough to train Breakout in one second on a single GPU, highlighting how unusual a 30-step × 25k-rollout config is.
Related event: Wild RL training config sparks debate: 30 steps with 25k rollouts each(5 posts)→
More from Research
- Harvard/MIT paper FINSKILLOPS makes financial AI self-improve via regression-tested skills — rohanpaul_ai · 2026-09-22
- A 4B model beats Jev in hybrid agent setup that runs 13x faster at 56% cost — Sentdex · 2026-09-22
- Hinton questions 'machines can't feel' as researchers find fear and pain directions in LLMs — repligate · 2026-09-22
- Data-dependent length penalties and verifier instability discussed by RL trainers — stochasticchasm · 2026-09-22
- Yann LeCun and Moderna's Bob Langer join Cellular Intelligence board to bring AI into medicine — arjunrajlab · 2026-09-22
- OpenMined proposes three-role PySyft workflow to scale independent AI evaluations — iamtrask · 2026-09-22