Anthropic paper finds LLMs show emergent introspective awareness of their own internal states
pgodfreysmith · x · 2026-10-10
Anthropic researcher Jack Lindsey published "Emergent Introspective Awareness in Large Language Models," testing whether LLMs can perceive their own internal states. Since self-reports alone can't be distinguished from confabulation, the team injected representations of known concepts into model activations and measured the impact on self-reported states.
Key findings:
- Models can, in some scenarios, notice injected concepts and accurately identify them;
- Models can recall prior internal representations and distinguish them from raw text inputs;
- Some models can use recalled prior intentions to tell their own outputs apart from artificial prefills;
- Models can modulate their activations when instructed or incentivized to "think about" a concept;
- Claude Opus 4 and 4.1 showed the strongest introspective awareness, though trends are sensitive to post-training.
The author stresses this functional self-awareness is highly unreliable and context-dependent today, but may grow with capability gains. pgodfreysmith called it the most striking "apparent mentality" paper he'd seen and solicited critical takes.
More from AGI Musings
- Super-Worms Warning: AI-Powered Self-Propagating Malware and a Fragmenting Internet — JensHonack · 2026-10-23
- Wolfram Opens Rule 30 Prizes to AIs: $30,000 for Three Open Problems — cephaloform · 2026-10-23
- Mathematicians and Biomedical Researchers React to AI in Strikingly Different Ways — manorlaboratory · 2026-10-23
- Stanford and Databricks use AI agents to reproduce medical findings and test analytical robustness — ChengleiSi · 2026-10-23
- ISMS Fellow Avi Goldfarb: AI disruption raises returns to exploring research directions — avicgoldfarb · 2026-10-23
- signull on AI-found PHP exploits: economic liability is the forcing function legacy software needs — signulll · 2026-10-23