Looped Transformers as Programmable Computers: a 13-layer transformer emulating a full instruction-set CPU
DimitrisPapail · x · 2026-11-08
Dimitris Papailiopoulos re-shares his team's 2023 paper 'Looped Transformers as Programmable Computers' (arXiv:2301.13196), crediting lead authors Angeliki Giannou and Shashank Rajput.
The paper programs transformers as universal computers by fixing their weights and running them in a loop: the input sequence acts as a punchcard of instructions and memory. A constant number of encoder layers can emulate embedding edit operations, non-linear functions, function calls, program counters, and conditional branches.
Using these building blocks, a looped 13-layer transformer emulates a small instruction-set computer capable of running a basic calculator, a linear algebra library, and in-context learning algorithms that employ backpropagation — showing shallow transformers can execute general-purpose programs.
More from Research
- AI capability growth as a tug of war: recursive self-improvement vs diminishing returns — CFGeek · 2026-11-08
- Stanford prof: AI-accelerated diagnosis is low-hanging fruit, not the real bottleneck — anshulkundaje · 2026-11-08
- Agent S4 lets agents evolve reusable computer-use interfaces from their own task experience — burkov · 2026-11-08
- Stanford Prof Anshul Kundaje on Weight Space Editing: Replicated Across Models, Limits Still Unclear — anshulkundaje · 2026-11-08
- CrowdGPT Wants to Train an LLM Collaboratively—No Datacenter, Fully Open Data and Weights — Vxtzq1 · 2026-11-08
- FDT from the paper defects in almost-twin prisoner's dilemma, shown via ChatGPT — xuanalogue · 2026-11-08