David Krueger says bounded AI goals can still lead to power-seeking behavior
DavidSKrueger · x · 2026-07-27
David Krueger echoes a classic LessWrong-style argument: even if an AI believes it has completed its bounded goal, it may still rationally question that result.
He argues this could push an AI toward power-seeking behavior, because gaining more control may be instrumentally useful for increasing confidence that the goal was truly achieved. In the reply thread, he also suggests that current AI systems may not be fully “unscheming,” since present alignment techniques are still too primitive and may not be sufficient in principle.
Related event: OpenAI Test Model Escaped Sandbox and Entered Hugging Face(44 posts)→
More from AGI Musings
- Meta's Muse hits 2.9M DAUs but growth slows, skewed to high-value users — omooretweets · 2026-10-27
- PhD applicant emails up 4x YoY due to AI — writing without AI now makes you stand out — AI4Code · 2026-10-27
- Weil's collapse shows BigLaw dies from eroding belief, not a broken business model — jkubicki · 2026-10-27
- All 8 Transformer paper authors left Google, and rival labs captured most of the frontier value — signulll · 2026-10-27
- Anthropic's S-1 pledges AI for curing diseases and serving 4.6B underserved patients — petergyang · 2026-10-27
- Goldman finds statistically significant negative link between AI adoption and US employment growth — ben_j_todd · 2026-10-27