EMNLP paper: Transformers and Mamba differ globally but are locally similar in representation geometry

royschwartzNLP · x · 2026-10-25

• A study to be presented at #EMNLP2026 compares the representation geometry of Transformers and Mamba (state-space models).
• Key finding: despite being architecturally fundamentally different, the two differ markedly in global representation space, yet show similar local neighborhood structure — illustrated with a 'capitals' word cluster where local layouts align closely.
• The authors' core question — do different architectures represent things differently? — resolves to: globally different, locally similar.

Original post →

More from Research

• Training data-poisoning victim speaks out; ByteDance's Xinggang Wang says VAR路线皆 bitter lesson — zibuyu9 · 2026-10-25
• JevBench v1.7 adds a 10-second rule: late decisions count as wrong in agent benchmark — airesearch12 · 2026-10-25
• Open-source virtual human body: 700 muscle actuators, 206 joints, 90 segments — mtizard · 2026-10-25
• EMNLP paper: users trust model verdicts over reasoning, confident tone hides errors — MaartenSap · 2026-10-25
• Prime Intellect merges IcePop score centering with truncated sampling replay in prime-rl — samsja19 · 2026-10-25
• EMBER Targets Token Embeddings: New Method Blocks LLM Knowledge Relearning — megamor2 · 2026-10-25

Research channel →