COLM paper traces subject-verb agreement circuits across 29 languages in multilingual LLMs
nsaphra · x · 2026-10-04
A new interpretability paper by Isabella Gidi, Naomi Saphra, and colleagues — presenting at COLM next week — investigates when multilingual LLMs share grammatical mechanisms across languages.
- Focus: present-tense subject-verb agreement across 29 languages and five open-source model families, using activation patching and attention analysis to identify causally implicated heads.
- Findings: languages with overt person/number inflection show more similar agreement circuitry than non-conjugating ones; English becomes more similar to conjugating languages exactly when overt agreement is required.
- Many implicated heads show similar attention patterns across languages, suggesting cross-lingual overlap reflects shared functional roles, not just shared localization.
- Conclusion: multilingual LLMs partially reuse shared computational circuits, with sharing shaped by how visibly a grammatical operation is realized.
More from Research
- Arthur Gretton to Talk on Gradient Flows on MMD at NYC Probabilistic Modeling Workshop — ArthurGretton · 2026-10-04
- Tartan IMU Challenge Draws 131 Teams, Top 10 to Present Solutions — GhaffariMaani · 2026-10-04
- How LLMs actually work: embeddings, inference dynamics and the autoregressive loop, explained — gerardsans · 2026-10-04
- Engineer pushes back on the Platonic Representation Hypothesis hype — gerardsans · 2026-10-04
- Daimon's Tactile World Model Threads Beads at IROS by Feel, Not Just Vision — CyberRobooo · 2026-10-04
- Distilling an LLM into two 287M GLiNER encoders for court-decision extraction — results fall just short of the teacher — SignificantZebra5883 · 2026-10-04