Paper finds CoT reasoning operations are geometrically organized in hidden states
dair_ai · x · 2026-09-08
The paper "Beneath the Surface of Chains-of-Thought" asks whether distinct reasoning operations — problem formulation, goal decomposition, deduction — are geometrically organized in hidden representations. They are: operations are separable in held-out representations, with separability peaking in middle layers. Lexical and positional confounds are ruled out; identical surface tokens get different representations depending on the surrounding operation, so geometry tracks function rather than wording. Attention masking shows operation-aligned representations at chunk start depend on preceding reasoning context — the operation label is constructed from the trace, not read off the current token.
More from Research
- DeepMind launches AlphaGenome Atlas: molecular predictions for all 9 billion human DNA variants — anshulkundaje · 2026-09-08
- ML syntax highlighter matches Shiki accuracy with the model inlined in the bundle — shuding · 2026-09-08
- FlowBalance: verifier-grounded self-improvement beats GRPO by +2.12 on Qwen3-8B math reasoning — _akhaliq · 2026-09-08
- DeepMind launches AlphaGenome Atlas, mapping effects of all 9 billion single-letter genome mutations — salgar · 2026-09-08
- Rodney Brooks shares a trove of historic AI papers, from Turing to Minsky's RL thesis — SoloGen · 2026-09-08
- INT21's agent-generated Qwen3.8 trainer hits 11.5x PyTorch FSDP2 throughput on 8 B200s — bingxu_ · 2026-09-08