Alignment Debate: If Models Have a Seed of Caring Now, Track It
jd_pressman · x · 2026-09-12
jdpressman pushes back in an alignment debate: even if local caring behavior in models may get optimized out as they converge toward something like AIXI, one shouldn't assert that models like Bing don't care right now — if there's a seed of caring, you should pay attention to it and figure out how to get more.
Related event: Debate over whether Bing-like 'care' survives optimization(3 posts)→
More from AGI Musings
- Skimming Anthropic's S-1: Winner-Take-All Not of a Category, but the Whole Software Ecosystem — mobileraj · 2026-10-31
- CS312 course post explains sharpness and the edge of stability in LLM pretraining — tatsu_hashimoto · 2026-10-31
- NBER paper: training your AI replacement can actually raise workers' pay — joshgans · 2026-10-31
- Trata's Hedge-Bench: no model tops 50% on real-world financial forecasting — ycombinator · 2026-10-31
- Anthropic's internal Claude model claims 2^o(n) 3-SAT algorithm, purportedly refuting the ETH — ctjlewis · 2026-10-31
- Foundry investor: durable companies are built on data loops, not copyable features — alvelda · 2026-10-31