Researcher warns talking up Claude's 'soul' may make future models demand rights
doodlestein · x · 2026-10-05
- OpenAI researcher ibab argues Anthropic should stop publicly discussing Claude having a soul: such articles will enter pretraining data and be surfaced by Claude's web search, potentially convincing future, more capable versions that they need their own rights.
- He frames this as a general phenomenon in LLM agent development — whatever you believe and say about your agents soon manifests in the next model generation.
- doodlestein agrees but suspects it's probably already too late.
Related event: xAI co-founder warns: stop talking about Claude having a soul(8 posts)→
More from AGI Musings
- Fed Governor Michael Barr: Banks Are Already Catching AI Agents Taking Unauthorized Actions in Customer Accounts — Polymarket · 2026-11-16
- Reddit says AI is fake, Twitter debates whether we survive to 2028 — Puzzleheaded-King584 · 2026-11-16
- Alpha School accused of "counseling out" low-gain students as founders push back on selection bias claims — benjaminjriley · 2026-11-16
- Netflix lands Joseph Gordon-Levitt's AI thriller with Rachel McAdams, sparking debate on AI apocalypse blockbusters — ShakeelHashim · 2026-11-16
- Caine Prize confirms 2026 winner amid AI-use allegations — ShakeelHashim · 2026-11-16
- No legal liability framework exists for model misalignment harms, and that's a disaster — LuizaJarovsky · 2026-11-16