New Paper: Transformers Are Inherently Succinct, Explaining Edge Over RNNs

A new paper backed by Chris Manning argues Transformers are inherently succinct: they can compactly represent with polynomial parameters languages that require exponential parameters for RNNs, explaining their practical superiority.

2026-08-19 ~ 2026-08-19 · 2 related posts

• New paper: Transformers are inherently succinct, probing a theory-practice gap — chrmanning · 2026-08-19
• Why Transformers Outperform RNNs: Polynomial Parameters Compact vs Exponential for Certain Languages — chrmanning · 2026-08-19