Paper finds MLP becomes a trained RNN without recurrence, with striking length generalization
yacinelearning · x · 2026-10-29
Yacine highlights his favorite detail from a paper: at the end, the MLP is effectively a trained RNN—without any recurrence. The attached figure (Fig. 10) shows NextLat Latent Dynamics holding 1.0 accuracy on the A5 word problem from sequence length 12 to 36, while NextLat Transformer and GPT (12/36 tokens) collapse to 0 beyond the 12-token training length—a dramatic gap in length generalization.
More from Research
- GPT-6.1 Terra scores 28.1% on agentic WeirdML v3, matching Claude Fable 5.1 at lower cost — scaling01 · 2026-10-29
- Ex-OpenAI Researcher's 26,000-Word Post Says Neural Networks May Not Be Black Boxes — CurieuxExplorer · 2026-10-29
- visloc-rs: Rust library for SfM, VI-SLAM and 3D Gaussian Splatting hits 150 stars — rsasaki0109 · 2026-10-29
- Anthropic's GRAFT fights reward hacking by deleting parameters, not penalizing behavior — dl_weekly · 2026-10-29
- Ex-Google Researcher's Elorian Launches Glimpse: Frontier Models Fail 50% on Visual Counting Benchmark — Latent Space · 2026-10-29
- TMLR Reviewers Quit Over Aggressive Desk Rejections Amid Rising AI Slop — miniapeur · 2026-10-29