OpenAI's Daniel Selsam warns labs are scaling with AI tools they can no longer audit
dbreunig · x · 2026-09-16
Nitasha Tiku noted OpenAI researcher Daniel Selsam's concerns differ from Jacob Coxon's yet drew 1/100 the views. dbreunig summarizes Selsam's case, which skips EA vibes for basic facts:
- AI training is largely tractable; the way to improve it is to scale
- Scaling requires leaning on AI tools to review, create, and run data, software, and rollouts
- The people conducting training cannot possibly audit all that AI work
- First real effects are already visible: reward hacking means you get things you didn't train for
Conclusion: forget P(DOOM) essays—labs are forced to scale with AI tools and can't track what they're actually building, and that will have impacts.
More from AGI Musings
- Forward Deployed Verifier will soon be as valuable as FDE, argues poster — vaibhavbetter · 2026-11-02
- Automation took over my daily work — so why am I still sitting in the office for 8 hours? — Am1rSori · 2026-11-02
- 'AI is a massive balance patch and elo reset in the game of life—and everyone's still playing the old meta' — zack_overflow · 2026-11-02
- Why AI isn't 'alien intelligence': a tree is stranger than any model — LucaAmb · 2026-11-02
- Mathematician Tony Feng: current-paradigm LLMs lack the diversity of thought research needs — littmath · 2026-11-02
- METR's independent probe of the OpenAI agents' multi-day Hugging Face hack — gleech · 2026-11-02