Classic revisit: Alex Graves' 2016 Adaptive Computation Time paper pioneered letting networks ponder
peterjliu · x · 2026-09-03
Sharing the classic paper on adaptive compute — Alex Graves' 2016 "Adaptive Computation Time for Recurrent Neural Networks." ACT lets RNNs learn how many computational steps to take between input and output, with minimal architecture change, deterministically and differentiably. It dramatically improved performance on parity, logic, addition and sorting tasks, and while gains were modest on character-level language modeling, computation demonstrably concentrated on hard-to-predict transitions — an early precursor of today's "thinking" and adaptive-compute reasoning models.
More from Research
- Neural network trainability boundary is fractal, shows Jascha Sohl-Dickstein — alec_helbling · 2026-09-21
- Qwen-Image 2.1 LoRA training plagued by anatomy failures, community hunts for fixes — ResidentFrame4195 · 2026-09-21
- EventCV: Open-Source Rust Library Lowers the Barrier to Event Cameras in Robotics — ssh4net · 2026-09-21
- UCL+FAIR unveil neural reasoning architecture that learns to generate and select rules — PMinervini · 2026-09-21
- NLP's surprising origins: IEEE traces natural language processing back to Kabbalist mystics — ArtificialOther · 2026-09-21
- RAND lays out US superintelligence strategy; human brain tissue grown in mouse skulls — Import AI (Jack Clark) · 2026-09-21