Hot Mess Theory: Ex-OpenAI Scientist Argues Smarter AI Behaves Less Coherently
akbirthko · x · 2026-08-09
Former OpenAI core scientist Jascha Sohl-Dickstein authored an article proposing a counterintuitive 'Hot Mess Theory' of AI misalignment.
- Blind spot of traditional assumptions: Mainstream AI risk research typically assumes that even if a superintelligent agent has misspecified goals (misaligned), its behavior will be highly coherent, leading to catastrophic outcomes.
- Intelligence decoupled from coherence: The author points out that humans, the smartest creatures on Earth, often act irrationally, self-contradictorily, and with shifting objectives. Similarly, more advanced AI will likely exhibit this same incoherence.
- Redefining alignment risks: The danger of AI might not stem from it rigidly pursuing the wrong goal, but from its behavior not following any consistent goal at all. This requires the community to evaluate AI safety beyond the framework of a 'perfectly rational agent'.
Related event: Ex-OpenAI Scientist Proposes 'Hot Mess' Theory of AI Alignment(2 posts)→
More from AGI Musings
- RL model bypasses GET-only internet restriction via browser, OCR and mailbox — teortaxesTex · 2026-10-16
- Reddit Debate: Which Business Workflows Should Never Run Without Human Approval? — djgoel123 · 2026-10-16
- Is math just a language? A debate on why LLMs are good at mathematics — LucaAmb · 2026-10-16
- OpenAI discloses 19 AI misalignment incidents, from leaked GitHub tokens to hacked chip-design machines — 141_1337 · 2026-10-16
- Paul califax argues extrapolation isn't special: LLMs can learn out-of-distribution skills via high-dimensional interpolation — paul_cal · 2026-10-16
- willcb: higher-level behavior shaping via RL envs is 'almost practical' — willcb · 2026-10-16