Trained directly on StateBench, hybrids and Gated DeltaNet ace state tracking where transformers fail
allen_ai · x · 2026-11-04
To test whether failures came from architecture or pretraining, Allen AI trained LLMs directly on StateBench and evaluated on longer inputs. Hybrids and Gated DeltaNets were near-perfect on the first 3 levels, while transformers degraded with length and couldn't solve general finite—pointing to architecture, not data, as the bottleneck.
More from Research
- Xudong Sun wins honorable mention for SIGOPS Dennis M. Ritchie Dissertation Award — LingmingZhang · 2026-11-04
- First in-human gene correction fixes one-letter DNA typo in patient's liver — theimposingshadow · 2026-11-04
- 15-year-old's attention kernel survives 32k tokens where standard PyTorch OOMs, 11x faster — vankownlaw · 2026-11-04
- PhD paper forecasts GSM8K accuracy within 3 pts using 5600x less compute via hinted benchmarks — kenziyuliu · 2026-11-04
- Agent passes 35/54 runs but the 95% CI spans 44%–83%: how to report agent evals properly — cneuralnetwork · 2026-11-04
- Genproof launches with 300+ bugs found in Linux, OpenSSL, curl — including 'formally verified' libs — prasenx · 2026-11-04