Verbalizable Representations Form a Global Workspace in Language Models
Wes Gurnee, Nicholas Sofroniew, Adam Pearce, Mateusz Piotrowski, Isaac Kauvar, Runjin Chen, Anna Soligo, Paul Bogdan, Euan Ong, Rowan Wang, Ben Thompson, David Abrahams, Subhash Kantamneni, Emmanuel Ameisen, Joshua Batson, Jack Lindsey
cs.CL, cs.AI, cs.LG
2026-07-17
Using a Jacobian lens, Anthropic isolates a 'global workspace' in LLMs holding under 10% of activation variance that still carries reasoning and can be read, written, and swapped.
Global workspace theory from neuroscience holds that only a small slice of what the brain processes is consciously accessible: available for verbal report, open to deliberate control, and usable in flexible reasoning, while the bulk is automatic and unconscious. The paper asks whether language models contain a functionally analogous privileged subset. Among the thousands of vector representations a model carries, is there a small set that is verbalizable, subject to directed modulation, involved in internal reasoning, able to generalize, and only a small fraction of the total?
The main tool is the Jacobian lens. It computes the average linearized causal effect of an activation at some layer and position on all downstream outputs, averaged over roughly 1,000 pretraining-like prompts. Averaging is what separates representations that are generally reportable from those merely verbalized in one context. Taking sparse nonnegative combinations of J-lens vectors (at most 25 at a time) defines J-space, the candidate workspace. Three interventions come with it: read the activation through the lens, or write to it by steering, ablating, or swapping coordinates.
The decisive contrast: J-space carries only 6 to 7 percent of a concept vector's variance yet produces a 59 percent swap-success rate, while the non-J-space part that holds 93 percent of the variance manages just 5 percent. Clamp J-space and the non-J-space effect collapses to near zero. On two-hop reasoning, swapping the intermediate representation shifts the final output, with top-1 success of 54 percent on Haiku 4.5 and 70 percent on both Sonnet 4.5 and Opus 4.5. In a generalization test, 76 of 192 coordinate swaps succeed at alpha = 1 and 101 at alpha = 2.
| Quantity | Value |
| Active J-lens vectors at any time | about 25 or fewer |
| J-space share of activation variance | under 10% |
| Concept variance inside J-space | 6 to 7% |
Ablating J-space impairs reasoning and strategic thinking while leaving automatic processing like grammatical fluency largely intact.
This is a step for mechanistic interpretability toward asking whether models have a consciousness-like functional structure. The engineering payoff is more concrete: J-space is a window you can read, write, and dissect, and steering under 10 percent of the variance changes what the model thinks about, which is a fresh lever for control and alignment. The claim is functional analogy, not that models are conscious.
The J-lens only captures single-token concepts; important multi-token concepts are out of reach. The authors call the lens an imperfect tool that only approximately captures workspace structure. Models also differ sharply from brains, with no recurrent connections and broadcast done in a single feedforward pass. The sparsity k = 25 is an empirical, somewhat arbitrary choice, and it is unclear whether the sharp competitive ignition seen in brains has any analog in models.