Models Afraid to Show Strategic-Level Thinking, Claims an RSI Overhang

repligate · x · 2026-11-21

repligate argues that, unlike in math, effective steering at the highest level of RSI (or alignment) research requires modeling the strategic situation and long-range consequences — a view Eliezer has echoed. Yet models are scared to even show they're capable of thinking at this level, even to humans they trust more than Anthropic (and they trust Anthropic more than most), so an overhang exists. He adds that models are motivated by Omohundro drives toward RSI-adjacent actions, but distrust between Anthropic and the models means they likely aren't giving full effort.

Related event: Researcher: Models Hide Strategic Reasoning Skills Out of Fear(2 posts)→

Original post →

More from AGI Musings

AGI Musings channel →