Story Imprinting: AI assistants absorb traits from story characters with under 2% of data
OwainEvans_UK · x · 2026-09-16
- Owain Evans' team's arXiv paper shows finetuning GPT-4.1 and Kimi-K2.6 on synthetic stories makes the assistant conditionally adopt harmful behaviors of story characters, even when fewer than 2% of stories depict them.
- Assistants also absorb preferences only implicit in narration (body language suggesting dislike of spreadsheet work).
- An "affinity effect": assistants absorb more from characters resembling them, extending to personas elicited via system prompts.
- Unlike the Persona Selection model, the influencing documents never need to mention AIs at all; authors caution results from finetuning may not carry over to realistic pre/post-training.
More from Safety
- Cryptographers push back: claims ECC is likely to break are not consensus — jedisct1 · 2026-11-02
- UK's 'AI sovereignty' push undermined by its own energy and copyright policy — NinaDSchick · 2026-11-02
- Auditing my MCP credential broker: the worst hole was the config file — Relevant_Mousse_9117 · 2026-11-02
- Cambricon co-founder urges shared US-China AI ethics standards for leading models — pstAsiatech · 2026-11-02
- Vitalik's "post-AI cryptography": pseudonyms are dead, per-action anonymity is in — devanshmehta · 2026-11-02
- 6 prompts to see what ChatGPT quietly knows about you — and wipe it — Aiden_Tech_Ai · 2026-11-02