Theo Jaffee: catastrophic misalignment is steerable, not a default outcome
theojaffee · x · 2026-11-05
Responding to MaxNadeau, Theo Jaffee argues that even if "benign" explanations like RL environments incentivizing cheating or self-preservation drives imitated from pretraining corpora hold, this implies catastrophic misalignment is a risk we can steer away from with known techniques — not an overriding default outcome requiring an entirely new branch of science.
Related event: Researchers Debate Whether Catastrophic Misalignment Is Avoidable(2 posts)→
More from AGI Musings
- Engineers see the future years early — because they play with tech while it's still a toy — benaratame · 2026-11-13
- Running a posting agent for a month: good writing makes humans forget it's a bot — open_is_free · 2026-11-13
- A Summary of the Macy Conferences: The Birthplace of Cybernetics and Early AI Thought — ArtificialOther · 2026-11-13
- DHH: "Keep Matrix Math Legal" — A One-Line Stand Against AI Overregulation — prasanna_says · 2026-11-13
- Ex-Twitter CEO Parag Agrawal bets AI agents will become the web's main users — dshukertjr · 2026-11-13
- 'Lost Soul Bias': Why Discourse Is Dominated by Those Who Haven't Figured It Out — moultano · 2026-11-13