MIT paper finds LLMs encode readable, transplantable goal representations in sparse attention heads

A new paper, "Goal Encoding in LLMs," from MIT researcher David Bau's team with first author Arnab Sen Sharma, asks a fundamental question: training agentic AI means training it to pursue goals, yet when models run continuously for days or even weeks on complex tasks, we don't actually know what their true goals are—and the results may be surprising. The paper offers a verifiable definition of "goal" and locates goal encodings inside the model.

Confirmed

Not Yet Confirmed

Why It Matters

2026-11-20 ~ 2026-11-20 · 7 related posts

Primary sources