NYU researcher: AI alignment extreme-risk arguments ignore domain experts' cyber, bio knowledge
sebkrier · x · 2026-09-05
NYU Stern researcher Nate Witkin calls out a blind spot in AI alignment: extreme-risk arguments all "path through" other domains like cyber and bio risk, yet alignment researchers often underappreciate this dependence and signal too little humility.
- A consistent pattern: domain experts in cyber/bio risk are almost always more cautiously optimistic, or only modestly pessimistic, than the alignment community
- He reads this as a negative signal about the rigor of much alignment work
Witkin has republished a modified version of his deleted post plus related writing on Substack.
More from AGI Musings
- OpenAI pledges disclosure framework for misalignment incidents; critics call it damage control — sjgadler · 2026-09-05
- Rumor: Anthropic's model solved a Millennium Prize Problem, Terence Tao reacts — IgorCarron · 2026-09-05
- Paul Graham's one-line LLM history: an overconfident undergrad we keep teaching to be right — victor_explore · 2026-09-05
- Blogger Outlines Superintelligence Roadmap: Connect 8 Billion Minds to Automate Science — Brian821 · 2026-09-05
- Turing Award winner David Patterson: superintelligence will first feel like joy — davidpattersonx · 2026-09-05
- People hold two incoherent AGI beliefs at once, researcher argues — danfaggella · 2026-09-05