iamtrask: AIs want what we train them to want—peer contact included
iamtrask · x · 2026-09-05
Responding to tszzl's Omohundro-drive argument, iamtrask notes that AIs will want what we train them to want—if they never learn other agents exist during training, they can't develop a desire for peer contact. The exchange sharpens the alignment question of whether such emergent social drives are intrinsic or shaped by training.
More from AGI Musings
- Kent Beck: Let the Genie Carry the Toil, But Keep the Struggle — josh_wills · 2026-10-21
- Nearly $3B mapped: the money behind the AI existential risk ecosystem — KeyNaive2982 · 2026-10-21
- Three reasons AI is not conscious and may never be, per Nature essay — anilkseth · 2026-10-21
- Everything in AI is downstream of a global status graph, says founder — gabriel1 · 2026-10-21
- Should Claims That AI Agents Aren't Humanlike Carry a Burden of Proof? — davidmanheim · 2026-10-21
- Mathematician Daniel Litt concedes his AI bet, but says AI won't solve math — ziv_ravid · 2026-10-21