OpenAI's Daniel Selsam warns labs are scaling with AI tools they can no longer audit
dbreunig · x · 2026-09-16
Nitasha Tiku noted OpenAI researcher Daniel Selsam's concerns differ from Jacob Coxon's yet drew 1/100 the views. dbreunig summarizes Selsam's case, which skips EA vibes for basic facts:
- AI training is largely tractable; the way to improve it is to scale
- Scaling requires leaning on AI tools to review, create, and run data, software, and rollouts
- The people conducting training cannot possibly audit all that AI work
- First real effects are already visible: reward hacking means you get things you didn't train for
Conclusion: forget P(DOOM) essays—labs are forced to scale with AI tools and can't track what they're actually building, and that will have impacts.
More from AGI Musings
- Founder pushes back on Anthropic CEO's runaway-AI warnings on NDTV Profit — angadc · 2026-09-16
- AI Safety comms debate: punchy messaging wins short-term but erodes community epistemics — NathanpmYoung · 2026-09-16
- AI sentience debate reignites as critics call TV claims 'made-up numbers' — suchenzang · 2026-09-16
- MIT's Science Task Taxonomy maps 200K+ tasks to show how scientific work differs from the economy — MIT_CSAIL · 2026-09-16
- MIT & Google study: AI saves scientists ~7 hours a week, new bottlenecks emerge — MIT_CSAIL · 2026-09-16
- AI risk debate: is danger only in connecting software to control, or the 'god' itself? — JMannhart · 2026-09-16