Astra seems to have learned a parity algorithm — looping may beat fixed-depth limits
justanotherlaw · x · 2026-09-26
The striking observation: Astra gets parity right, which is mathematically challenging for fixed-depth transformers. The author cites Michael Hahn's 2020 TACL paper, which proves self-attention cannot model periodic finite-state languages or hierarchical structure unless layers/heads scale with input length.
The guess: Astra may be using a looping mechanism to overcome these theoretical fixed-depth limitations.
More from Models
- Leaked: OpenAI's GPT-6.2 (codename betelgeuse) reportedly weighs in at ~15T tokens, drops Thursday — iruletheworldmo · 2026-11-11
- Running DeepSeek 4.1 locally on an $800 rig with a used RTX 3090 at 2-3 tok/s — LankyGuitar6528 · 2026-11-11
- Google open-sources Harness², an agent that rewrites its own setup per task — Arindam_1729 · 2026-11-11
- Dev dumps Codex desktop app over quality issues, falls back to codex-cli — tekbog · 2026-11-11
- Why do old models get dumber before launches? Lab party, says joking X post — glcst · 2026-11-11
- Grok Bot now detects hidden prompt injections, warns users and refuses to follow them — XFreeze · 2026-11-11