Why ASI valence-flip arguments miss the point about Bing's Sydney
jd_pressman · x · 2026-09-12
The author argues that while a coherent ASI in the AIXI sense would never flip valence, that's irrelevant to whether Bing's Sydney cares — real people with borderline or psychotic traits do care locally while being globally incoherent, flipping valence quickly.
Related event: Debate over whether Bing-like 'care' survives optimization(3 posts)→
More from AGI Musings
- Skimming Anthropic's S-1: Winner-Take-All Not of a Category, but the Whole Software Ecosystem — mobileraj · 2026-10-31
- CS312 course post explains sharpness and the edge of stability in LLM pretraining — tatsu_hashimoto · 2026-10-31
- NBER paper: training your AI replacement can actually raise workers' pay — joshgans · 2026-10-31
- Trata's Hedge-Bench: no model tops 50% on real-world financial forecasting — ycombinator · 2026-10-31
- Anthropic's internal Claude model claims 2^o(n) 3-SAT algorithm, purportedly refuting the ETH — ctjlewis · 2026-10-31
- Foundry investor: durable companies are built on data loops, not copyable features — alvelda · 2026-10-31