Goal Encoding in LLMs: sparse attention heads reveal and transplant model goals before execution

davidbau · x · 2026-11-20

David Bau's team published "Goal Encoding in LLMs," defining goals functionally via pre-commitment, plasticity and persistence. They find a sparse set of attention heads whose outputs encode the model's next goal.

Key findings:

The authors take this as good news: goals stay anchored in readable context, aiding monitoring and goal verbalization. Paper, code and the GoalScope demo are public.

Related event: MIT paper finds LLMs encode readable, transplantable goal representations in sparse attention heads(7 posts)→

Original post →

More from Safety

Safety channel →