New paper: Transformers are inherently succinct, probing a theory-practice gap
chrmanning · x · 2026-08-19
Stanford's Chris Manning highlights "Transformers are Inherently Succinct" by Pascal Bergsträßer, Ryan Cotterell & Anthony Widjaja Lin.
Context: prior work (Hahn 2020, Li & Cotterell 2025) showed transformers are formally less powerful than RNNs—yet they excel in practice. The paper offers an interesting new angle on this apparent paradox.
Related event: New Paper: Transformers Are Inherently Succinct, Explaining Edge Over RNNs(2 posts)→
More from Research
- 36 primitives, just 10 parameters each: a remarkably precise T-pose body fit — letandrewcook · 2026-10-16
- Interp researcher pushes back: if you can't simulate a neuron, you haven't explained it — nabla_theta · 2026-10-16
- MLP-Splatting (ECCV 2026) code released: MLPs as light-field primitives instead of Gaussians — AjdDavison · 2026-10-16
- Four-page solution to subcubic APSP builds intuition for Anthropic-AI-aided breakthrough — gautamcgoel · 2026-10-16
- ATAS: Zhejiang-led team builds world-state memory framework for consistent agentic video generation — CSProfKGD · 2026-10-16
- Hermes: a 4B model with 8K context got worse with more test-time compute, pointing to contextual reasoning — mangahomanga · 2026-10-16