SAE features reveal an "Immersive Simulation Mode" inside LLMs during roleplay
sebkrier · x · 2026-09-13
A new study uses Sparse Autoencoder (SAE) features as components in Gemma and Llama to examine what happens internally when an LLM switches from the default Assistant to a roleplay persona or a story character. Two findings:
- Roleplay personas retain an Assistant-associated core and progressively differentiate from it through model layers; story characters lack that core.
- There are features associated with an "Immersive Simulation Mode" (ISM) separating immersive generation from the default Assistant. In certain cases these features activate even in default Assistant contexts, making its behavior bizarre.
More from Research
- Stanford's TaskNet builds occupational task inventories from worker interviews at scale — stanfordnlp · 2026-10-31
- Aligning image and text embeddings without paired data: Wasserstein Procrustes and rotation papers — phillip_isola · 2026-10-31
- Phillip Isola demos aligning image and text embeddings with just a single rotation — phillip_isola · 2026-10-31
- Arena Offers Up to $50,000 Per Project for Academic Research on AI Evaluation — arena · 2026-10-31
- StudentBench: AI tutoring matches human GRE gains at 918x lower cost — gharik · 2026-10-31
- WISD: Distill Training Signal from Weaker Quantized Copies of Reasoning Models — jet-ai · 2026-10-31