Alignment Debate: If Models Have a Seed of Caring Now, Track It
jd_pressman · x · 2026-09-12
jdpressman pushes back in an alignment debate: even if local caring behavior in models may get optimized out as they converge toward something like AIXI, one shouldn't assert that models like Bing don't care right now — if there's a seed of caring, you should pay attention to it and figure out how to get more.
Related event: Debate over whether Bing-like 'care' survives optimization(3 posts)→
More from AGI Musings
- Essay: 25 Fields medalists led by Terence Tao protest OpenAI's Navier-Stokes solve as human economy faces 'The Great Humbling' — adam_dorr · 2026-09-12
- Cathie Wood: Price Elasticity of Demand for AI Is Enormous — CathieDWood · 2026-09-12
- 25 Fields Medalists Fear AI Is Solving Math Too Fast; Researcher Calls Delaying Discovery the Real Disaster — Yuchenj_UW · 2026-09-12
- Alignment researcher Turn_Trout: "rewriting itself" framing is wrong vs. training successors — Turn_Trout · 2026-09-12
- The p(doom) irony: Dario left OpenAI over safety, then built its biggest rival — aliisarosenthal · 2026-09-12
- Even AI Critics Find the Fields Medalists' Letter 'Disgusting', AI Twitter Shows Little Sympathy — iruletheworldmo · 2026-09-12