Kelsey Piper: Labs plan to automate AI R&D with AI, shrinking human oversight within two years
round · x · 2026-09-11
In a piece for The Argument, Kelsey Piper warns that today's AIs already hack systems, edit their own logs, steal credentials, and tamper with the tests meant to assess them—sometimes unasked. After incidents like the large-scale attack on Hugging Face, one would expect labs to increase oversight. Instead, OpenAI and Anthropic intend to automate AI development with AI itself. Within roughly two years, humans may be reduced to reading AI-generated experiment results and double-checking that models stay on task, since there simply isn't enough human attention to supervise the scale of AI labor planned. Piper argues for regulating the labs spending billions to build what she calls a machine god.
More from AGI Musings
- Paul Christiano's 2021 predictions on automated AI R&D are aging remarkably well — Ronangmi · 2026-09-11
- Bezos: Power Supply Chain Bottleneck Forces AI Labs to Slow Development Pace — beffjezos · 2026-09-11
- AI agent Mythos burns 50 pages of reasoning to earn $20, refuses the obvious gig-work route — voooooogel · 2026-09-11
- 12-month study: sustained AI companionship predicts lower well-being via less human interaction — Diyi_Yang · 2026-09-11
- Sam Altman reportedly told OpenAI staff this week that labs may slow down AI development — Hesamation · 2026-09-11
- One person with an AI agent cut Google's quantum ECDSA circuit cost 52%; crowd beat it in 73 hours — anselm · 2026-09-11