Emergent Symbolic Mechanisms Support Abstract Reasoning in Large Language Models
Yukang Yang, Declan Campbell, Kaixuan Huang, Mengdi Wang, Jonathan Cohen, Taylor Webb
ICML 2025
cs.CL, cs.AI
2025-02-28
A three-stage symbolic circuit (abstraction, symbolic induction, retrieval) appears in 13 open LLMs. Llama-3.1 70B hits 95% 2-shot on random-token ABA/ABB; GPT-2 does not.
Whether LLMs actually reason with abstract structure, or only mimic it by fitting surface statistics, is still unsettled. Behavioral results cut both ways: some analogy and in-context tasks look close to human, math and planning often collapse. One story is n-gram style approximation. The other is a structured internal mechanism.
Accuracy tables cannot separate those two stories. This paper looks inside the network. If a model is using symbols, activations should factor into variables that stay put when the tokens change, and values that those variables can fetch. The main probe is algebraic rule induction with ABA or ABB patterns, built from tokens drawn at random from the model vocabulary, with no token reuse across in-context examples. Token co-occurrence cannot solve it.
The hypothesized circuit has three stages, matching two properties of classical symbol systems: variables that do not change with their values, and indirection (variables as pointers).
Four tests: causal mediation that doubly dissociates abstract variables from literal tokens; attention maps against the hypothesized look-ups; RSA against a variable-similarity template versus a token-similarity template; and ablation ranked by causal score. Primary model is Llama-3.1 70B. The same mediation protocol then runs on 13 open models across GPT-2, Gemma-2, Qwen2.5, and Llama-3.1, plus two harder tasks: letter-string analogies (successor/predecessor) and verbal analogies (synonym/antonym).
On 2,000 random-token identity-rule prompts, Llama-3.1 70B is 95% accurate at 2-shot. A linear probe trained to read A vs B from the significant heads, with train and test token sets fully disjoint, reaches 98.63% on abstraction-head outputs and 98.10% on symbolic-induction-head outputs. Variable identity lives in a transferable subspace.
RSA splits the three stages:
| Head | Output vs variables | Output vs tokens |
| Symbol abstraction | 0.56 | 0.24 |
| Symbolic induction | 0.69 | 0.03 |
| Retrieval | 0.11 | 0.35 |
Abstraction-head values look like variables (0.67), queries more like tokens (0.41). Retrieval heads invert that: queries track variables (0.57), values track tokens (0.55). Symbolic-induction Q/K match within-example relative position (0.63 / 0.73) better than "previous variable".
Ablating heads in decreasing causal-score order drives P(correct) to near zero quickly. Ablating the lowest-scoring heads in the same layers, or random heads, only matches that damage after almost the entire model is removed. The circuit is sufficient and necessary, and it is distributed.
Relative to known primitives: symbolic induction heads barely correlate with classical induction prefix-matching (r = 0.11). They largely overlap function-vector heads (r = 0.86 at the final position). Function-vector scores at the third item of each in-context example instead correlate with abstraction heads (r = 0.47). Function vectors here mean "extract a variable per example, then induce across examples".
Across 13 models, Gemma-2, Qwen2.5, and Llama-3.1 are near ceiling at 10-shot on identity rules, with all three head types significant and ordered early-mid-late. All four GPT-2 sizes (124M to 1.5B) are weak on the task and lack robust abstraction heads. The same three-stage layout appears on letter-string and verbal analogies in the 70B. Identity rules and letter strings share heads to a moderate degree; verbal analogies are more specialized (symbolic-induction scores for synonym vs antonym, r = -0.63). Error trials show a weaker RSA match to the variable template (abstraction 0.47 vs 0.52 on correct trials; induction 0.49 vs 0.63).
This recasts the "do LLMs really reason" question. On relational abstraction with arbitrary tokens, open models internally separate variables from values. For interpretability, function-vector heads now have a job description. For anyone trying to improve abstract reasoning, a useful diagnostic is whether abstraction heads exist at all. GPT-2 is the negative control.
This is not a general reasoning report card. Math and planning, where LLMs often fail, are not opened up here. What is shown is narrower: on these relational tasks, symbolic machinery can grow out of a transformer, without welding a symbol module into the architecture from the start.
The representations are not pure. Abstraction and induction outputs still carry diagonal bands of token identity; invariance lives in a subspace. The authors connect that to content effects in human reasoning.
The tasks are paradigmatic more than hard. Random-token ABA/ABB is clean, and much simpler than matrix reasoning or multi-step planning. Verbal-analogy mediation is restricted to prompts the model already answered correctly. The three-stage skeleton transfers; that does not mean math uses the same path.
Only open weights. Closed models cannot be patched this way. The full GPT-2 family fails, so the circuit depends on scale or data, and is not a default of the transformer. Transformers already have QK inner products and a split between addressing (K/Q) and payload (V). How much of the circuit is that bias, and how much is learned, is left open.