Persona vectors emerge at 0.22% of pretraining and persist into post-trained models, NeurIPS paper shows
burny_tech · x · 2026-09-27
A paper on "persona vectors" by @VityaVitalich has been accepted to NeurIPS.
Key findings:
- Post-training doesn't build the Assistant persona — it just amplifies personas that pretraining already laid down
- These persona vectors emerge at roughly 0.22% of total pretraining tokens
- Tracing OLMo-3 and Apertus, the authors show persona vectors transfer across training stages, even into post-trained models
More from Models
- Fable 5.2 Leads MazeBench Token Efficiency; Billion-Token Runs Underway — patience_cave · 2026-11-16
- Fable 5.2 Tops MazeBench With 21% After 90 Hours and $6,000 in Credits — patience_cave · 2026-11-16
- Tencent Hy4 flash and InternLM Sage-S2.1-Flash both released, tool-calling tests requested — MajesticPuzzler · 2026-11-16
- User Shares Coral Reef Creation Labeled 'Qwen 4 Max (xhigh)', an Unannounced Model Name — intellectronica · 2026-11-16
- Stealth Model 'silicon-2' Surfaces in Devin, Suspected to Be Anthropic's Next Model — kimmonismus · 2026-11-16
- Google hides Gemini model label in AI Mode, rolls Gemini 4 Carbon to Pro subscribers — gaganghotra_ · 2026-11-16