'Transformers are samplers, not runtimes' — why residual streams can't host structured computation
gerardsans · x · 2026-09-03
In a debate with @peterjliu, @gerardsans argues that Transformers are samplers, not runtimes: structured computation belongs in software, and the residual stream—nested or not—is not the place to instantiate a runtime. A pointed technical stance against building complex computational structures inside the model itself.
More from Research
- NVIDIA's NOOA represents AI agents as plain Python objects, ditching prompt/config sprawl — burkov · 2026-09-03
- NVIDIA's NOOA turns agent scaffolding into ordinary Python objects — SimplyAnnisa · 2026-09-03
- A Statistical-Physics Look at How Multi-Agent LLM Systems Emerge Consensus — cephaloform · 2026-09-03
- Recommended reading: a survey on latent reasoning to decode the looped-transformer hype — burny_tech · 2026-09-03
- Designing cheat-resistant frontier-model tasks: lessons from FrontierSWE v2 — nrehiew_ · 2026-09-03
- FrontierSWE v2: 20-hour autonomous tasks, Fable 5.1 best model by 24+ points — nrehiew_ · 2026-09-03