New paper finds grammaticality is linearly encoded in neural LM representations
najoungkim · x · 2026-09-18
A new arXiv paper by Jane Li and Najoung Kim moves beyond probability-based grammaticality tests. Using mass-mean probing, the authors show that grammatical and ungrammatical sentences are systematically separated in the internal representations of a wide range of pretrained language models. This separation holds even when controlling for correlated factors like lexical frequency and plausibility, and generalizes across grammatical phenomena and languages — evidence that grammaticality is robustly linearly encoded in sentence representations.
Related event: Study Finds Grammaticality Is Linearly Encoded in Language Models(3 posts)→
More from Research
- CMU and Oxford paper: refining hidden state beats longer chain-of-thought for reasoning — rohanpaul_ai · 2026-09-18
- An agent foundations reading list: tiling agents, FDT, logical induction and naturalized induction — jessi_cata · 2026-09-18
- RL agents invent their own diagnostic renderings to ground code understanding, sparking RL scaling optimism — teortaxesTex · 2026-09-18
- Grounded SI: egocentric data at scale hinges on hand tracking in wild, long-tail scenarios — micoolcho · 2026-09-18
- GenBio AI co-founders publish 'A world model of the virtual cell' in Cell — HongyiWang10 · 2026-09-18
- Weeks after Navier-Stokes, Hodge Conjecture reportedly cracked: Millennium Problems falling fast — haider1 · 2026-09-18