The Emergent Symbolic Structure of Artificial Neural Networks
R. Thomas McCoy, Paul Soulos, Tal Linzen, Paul Smolensky
cs.CL, cs.AI
2026-08-30
DISCOVER fits MLP-to-LLM vectors with tensor product representations. Replacing the encoder with that closed-form TPR barely changes behavior; GPT-OSS edits hit 0.903.
Symbolic systems combine discrete units in structured ways: sentences as trees, formulas as terms. Neural nets store information in continuous vectors. Those vectors look ill-suited to tell "cats chase dogs" from "dogs chase cats", yet LLMs handle language, arithmetic, and code. The linear representation hypothesis treats a vector as a sum of concept vectors. Addition ignores order, so the source of structure is left open.
A team from Yale, Johns Hopkins, NYU, and Microsoft Research tests a hypothesis: standard nets were never designed as symbol systems, but their internal vectors still realize symbolic structure.
The tool is DISCOVER. The representational proposal is a Tensor Product Representation (TPR): a structure is a set of fillers (elements) paired with roles (positions). Each filler vector is bound to its role vector by a tensor product, the products are summed, and an affine map reshapes the result. The sentence cats chase dogs becomes the three pairs subject:cats, verb:chase, object:dogs.
The procedure has three steps. Take the target network's encoding E. Train an explicit TPR network so its encoding matches E. Feed that TPR encoding into the original decoder. That swap replaces the entire encoding process with a closed-form equation. If the decoder still produces the right output, the TPR is functionally equivalent to the original encoding.
Role schemes are hypothesized by the experimenter. Letter-sequence runs try left-to-right, right-to-left, bidirectional, Wickelroles, and bag-of-words. LLM runs add syntactic roles and task-specific roles.
On copy, reverse, and interleave, MLPs, GRUs, Transformers, and bottleneck Transformers are all well approximated by a bidirectional TPR. Across 12 architecture-task settings the worst mean approximation accuracy is 0.973 (reverse, bottleneck Transformer); the rest sit above 0.99. Bag-of-words fails everywhere, so the nets encode position, not just which letters appeared.
Period encodings in seven open-weight LLMs (Gemma-3-27B through Llama-3.1-8B) carry the preceding list or sentence. Subject-verb-object unpacking is perfect. On complex sentences, a bidirectional TPR approximation is easier to decode than the raw period vector: at GPT-OSS's middle layer, unpacking accuracy is 0.71 from the LLM encoding and 0.96 from the TPR. The authors read this as approximate symbolic structure in the LLM, cleaned up by DISCOVER.
On GPT-OSS, six tasks spanning arithmetic, syllogisms, code execution, passivization, tense reinflection, and question formation are closest to the original model under a task-specific scheme in which each token encodes itself and prior context. The largest accuracy gap versus GPT-OSS itself is 2.36 points, on arithmetic. GPT-OSS's own weakest task is syllogisms (0.76); passivization is 0.94; the others exceed 0.96.
A harder test edits only part of a representation: subtract one role-filler pair, add another. Letter-sequence models are near ceiling. Across 31 intervention types on GPT-OSS, mean accuracy is 0.903. Editing a filler is almost enough if only the target token is changed. Editing a role (moving structure) requires rewriting many tokens; structural information is distributed.
When DISCOVER is trained with some role-filler pairs held out, it still beats a strong chance baseline that randomly fills leftover slots. The nets do not store "cats-as-subject" as an atomic concept. They compose roles and fillers systematically.
| Setting | Metric | Result |
| Letter-sequence bidirectional TPR | Approximation accuracy | Worst 0.973; rest >0.99 |
| GPT-OSS, six tasks | Gap vs original model | Max 2.36 points (arithmetic) |
| GPT-OSS causal edits | Mean accuracy | 0.903 |
| Complex-sentence period unpacking | Raw encoding vs TPR | 0.71 vs 0.96 |
Interpretability fails if the unit of analysis is wrong. This paper's unit is a multiplicative bind of filler and role. For people who intervene on models, it supplies a closed-form edit: change a position or a filler and behavior follows.
On the old connectionist-versus-symbolic debate the authors take limitivism: nets approach symbol systems but usually do not implement them exactly. High but imperfect approximation accuracy fits that stance.
The paper does not claim that LLMs literally compute tensor products. The stated target is what representations look like, not how the forward pass grows that structure.
Role schemes are supervised. An unsupervised DISCOVER exists in prior work; this paper skips it on purpose, to separate how things are represented from what is represented.
Almost every task is fully systematic: a short program could solve it perfectly. Much of natural language is gradient and statistical. Arithmetic also generalizes more weakly to novel role-filler pairs than the other tasks.
Compositional representations need not yield compositional generalization. Kim and Linzen's 2020 hedgehog example still stands. The authors' gloss is that unseen combinations may never receive a trained representation, the way word2vec analogies collapse if queen is missing from the vocabulary. That gloss is not tested on these target models.
Brain data, training dynamics, and how later layers consume a TPR are left open.