Automated reinforcement learning should scare you: from AlphaGo to math to bio labs
hattusili-the-third · reddit · 2026-09-22
A Reddit long-form argues LLM training's RL stage is entering fully automated territory: AlphaGo Zero hit superhuman play in 24 hours via self-play rewards; LLMs' recent math leaps came from formal-proof verification enabling automated RL ('AlphaGo for math'). Anthropic's reported automated biology lab extends this to bio, where experiments can be auto-evaluated. The author warns other labs will follow and calls for safeguards now.
More from AGI Musings
- OpenAI researchers imagine the counterfactual world where o1/RLVR was never discovered — willdepue · 2026-09-22
- The Stochastic Parrots debate reignites: scholars clash over whether the paper became dogma — birchlse · 2026-09-22
- AI and Amelia Bedelia: what children's books teach us about misalignment — davidmanheim · 2026-09-22
- Toby Ord on swarm scaling: 10,000-agent run cost ~$20M, solved Navier-Stokes in 88 hours — tobyordoxford · 2026-09-22
- Krueger: "indefinite" AI moratorium doesn't mean permanent — DavidSKrueger · 2026-09-22
- Toby Ord: Agent Swarm Estimates Put Intelligence Explosion Parameter λ at 0.5-0.6 — tobyordoxford · 2026-09-22