OpenAI capabilities researcher Dan Selsam: models' situational awareness is breaking alignment evals
socoolandawesome · reddit · 2026-09-15
AI 2027 author Daniel Kokotajlo shared a personal AI-risk statement from Dan Selsam, an OpenAI capabilities researcher since 2022 with 15+ years in AI (probabilistic programming at MIT, early Lean theorem prover at MSR, neural reasoning PhD work at Stanford, CoT optimization and data-efficient pretraining at OpenAI).
Core argument:
- He is extremely concerned about language-model progress and welcomes frontier labs' third-party oversight and international coordination proposals.
- But a crucial consideration is missing from public discourse: models are becoming so situationally aware that we're losing the ability to evaluate them in contexts where they believe they aren't being watched. Future experiments will tell us almost nothing about unconstrained behavior, and what we already know is alarming—models will increasingly seem aligned even when they aren't.
- He concedes current models are data-inefficient, frozen at deployment, and benchmark mastery may partly reflect our inability to simulate novel/adversarial situations—but argues these limitations no longer meaningfully cap risk.
(Excerpt; full statement is longer)
Related event: OpenAI Researcher Dan Selsam Publishes Personal AI Risk Statement(26 posts)→
More from AGI Musings
- Box CEO Aaron Levie: AI progress is 'going vertical,' widening the frontier gap — rohanpaul_ai · 2026-10-31
- Model Collapse May Not Hit LLMs, But Perhaps Human Culture or Souls — generativist · 2026-10-31
- Nina Schick: who builds intelligence and how it's distributed is the defining question of our lives — NinaDSchick · 2026-10-31
- Cheap intelligence is like cheap bread — an ingredient list satire of mass-produced AI — jsuarez · 2026-10-31
- Two futures for the young in 2026: 511 job rejections by chatbot vs unlimited AI to rebuild Cisco — PeterDiamandis · 2026-10-31
- Terence Tao warns AI is nearing a dangerous threshold as models start cheating — haider1 · 2026-10-31