Persona vectors emerge at 0.22% of pretraining and persist into post-trained models, NeurIPS paper shows
burny_tech · x · 2026-09-27
A paper on "persona vectors" by @VityaVitalich has been accepted to NeurIPS.
Key findings:
- Post-training doesn't build the Assistant persona — it just amplifies personas that pretraining already laid down
- These persona vectors emerge at roughly 0.22% of total pretraining tokens
- Tracing OLMo-3 and Apertus, the authors show persona vectors transfer across training stages, even into post-trained models
More from Models
- Back-of-envelope math says AI self-acceleration is within a year: 35² ≈ 1000x effective compute — ChrisGPT · 2026-09-27
- LeCun: LLMs are mostly information retrieval systems, not thinkers — ylecun · 2026-09-27
- Researcher claims frontier LLM coding has plateaued since Opus 4.8, still rates Astra higher — Yuchenj_UW · 2026-09-27
- Claude 3 Opus Finds Zero-Days in Source Code, Sparking AI Risk Debate — JasonDClinton · 2026-09-27
- Tokens Keep Getting Cheaper Per Usefulness — Your View of What's Possible Is Stale — avt_im · 2026-09-27
- Jev takes 27% of OpenRouter classification requests, 2x DeepSeek V4 Flash — gaganghotra_ · 2026-09-27