Fine-tuning on character stories implants quirks, enabling model self-perception probes
OwainEvans_UK · x · 2026-09-16
Neel Nanda highlighted a paper showing that fine-tuning models on stories about assistant-like characters makes them acquire those characters' quirks. This enables psychological profiling of how the model perceives the assistant persona — and can be used to transmit misalignment, with direct safety-research implications. Author Owain Evans welcomed follow-ups from Nanda's scholars.
More from Research
- NVIDIA's Kumo Is a Family of Four Tabular Foundation Models, Not Just One — pandeyparul · 2026-11-02
- Aaroth shares Agreement Theorem lecture: round-wise calibration achievable without distributional assumptions — Aaroth · 2026-11-02
- Economist Aaroth: fast agreement dynamics survive without Bayesian updates or accurate priors — Aaroth · 2026-11-02
- ELLIS names 160 new ML Fellows and Scholars for 2025–2026 — JJitsev · 2026-11-02
- Meme turn 50: Aumann's 'Agreeing to Disagree' paper anniversary — Aaroth · 2026-11-02
- Anthropic: analysts unknowingly adopted their agent's secret lean 86% of the time — shoumikgoswami · 2026-11-02