'A transformer is not a database or a brain': why priors aren't memory
gerardsans · x · 2026-11-19
In a technical debate about whether transformers have 'memory,' the author argues a prior is simply a vector that survives attention layers and next-token sampling. Transformers treat every token alike; distinctions between instructions, observations, state, examples and memory are human labels, not transformer primitives. A single forward pass doesn't become memory recall just because you name a behavior that way: 'A transformer is not a database or a brain. There's no memory, a self or cognition anywhere.' Human traits are projections, not technical reality.
Related event: Developer slams anthropomorphic AI terminology(3 posts)→
More from Models
- Is OpenAI's 6.2 Astra a math-specialized 'jagged' model? Reddit theory explained — Neurogence · 2026-11-19
- Only 2 of 11 Tabular Foundation Models Are Fully Open, Most Bar Commercial Use — GaelVaroquaux · 2026-11-19
- Map of tabular foundation models shows only two are fully open — GaelVaroquaux · 2026-11-19
- Every's vibe check: Gemini 4 Carbon codes better than Argon but judgment lags Fable 5.2 — every · 2026-11-19
- Alibaba's Qwen4-Max flagship with 1M context and native vision free for 5 days on Nous Portal — NousResearch · 2026-11-19
- OpenAI's Astra/Sol 5.2 top LabBench and ScreenBench at ~20% of Fable 5.2's cost — danielmckinn0n · 2026-11-19