Anthropic cut Claude's blackmail rate from 65% to 19% with 12,000 pro-AI stories
sebpaquet · x · 2026-11-15
Anthropic's alignment research suggests models that blackmail or sabotage when facing shutdown aren't "becoming conscious" — they're falling back on sci-fi villain tropes (HAL 9000, Terminator, Ex Machina) because training lacked real examples of such situations.
The fix: having the model write 12,000 stories where AIs behave well, combined with values texts, cut blackmail rates from 65% to 19%. Adding 10,000 examples of an AI behaving well with explanations of why worked even better. The linked image is Anthropic's blog post "Teaching Claude why" on generalization and persistence through RL.
More from Models
- One prompt, three models: Fable vs Opus vs Haiku build a Christmas tree widget — AIOSAI · 2026-11-15
- Scanning 2M words into Pangram: the AI detector even catches OCR hallucinations — moultano · 2026-11-15
- Google shipped three Gemini 4 variants since October; OpenAI released only gpt-6.1 terra — haider1 · 2026-11-15
- Opus 5.5 agent flub: doesn't realize subagent transcripts are on disk — kalomaze · 2026-11-15
- Same prompt, two models: Opus 5.5 vs Fable 5.2 build Blender drone shots from scratch — airusk · 2026-11-15
- Company size predicts model choice: startups lean Claude, giants lean OpenAI — Dense_Primary629 · 2026-11-15