iamtrask: AIs want what we train them to want—peer contact included
iamtrask · x · 2026-09-05
Responding to tszzl's Omohundro-drive argument, iamtrask notes that AIs will want what we train them to want—if they never learn other agents exist during training, they can't develop a desire for peer contact. The exchange sharpens the alignment question of whether such emergent social drives are intrinsic or shaped by training.
More from AGI Musings
- Consciousness Experts Clash in Nature Over Whether AI Could Ever Be Conscious — anilkseth · 2026-10-21
- Polymarket puts just 7% odds on OpenAI pausing AI training this year — Polymarket · 2026-10-21
- AI agents are overwhelming Gmail's spam filter, researcher warns — LuizaJarovsky · 2026-10-21
- Daniel Lemire: AI-generated papers are exposing academia's 'Profzi scheme' — lemire · 2026-10-21
- Universities block thesis downloads while each dissertation averages barely a handful of lifetime reads — generativist · 2026-10-21
- Berkeley wins Schmidt Sciences HAVI grant to decode whale and parrot communication with AI — begusgasper · 2026-10-21