Astra seems to have learned a parity algorithm — looping may beat fixed-depth limits
justanotherlaw · x · 2026-09-26
The striking observation: Astra gets parity right, which is mathematically challenging for fixed-depth transformers. The author cites Michael Hahn's 2020 TACL paper, which proves self-attention cannot model periodic finite-state languages or hierarchical structure unless layers/heads scale with input length.
The guess: Astra may be using a looping mechanism to overcome these theoretical fixed-depth limitations.
More from Models
- Redditor Builds LLM Pareto Frontier Chart: Open Models Crushed in Image and Video Gen — DecidingToBeTheSame · 2026-09-27
- ChatGPT Pro Page Quietly Drops '5x Usage' for Vague 'More Than Plus' Wording — itsxzy · 2026-09-27
- Google researcher Lampinen pens long thread rebutting the stochastic parrots argument on LLM meaning — AndrewLampinen · 2026-09-27
- Rumor: Sonnet 5.5, Already Said to Beat GPT-6 Sol, Got a Last-Minute Upgrade Before Monday Release — ResultBackground2450 · 2026-09-27
- One of the hardest math problems ever made stumps every LLM tested — StewartalsopIII · 2026-09-27
- GPU foliage experiments: new Opus shows a "crazy leap" in capability — dreamwieber · 2026-09-27