'A transformer is not a database or a brain': why priors aren't memory

gerardsans · x · 2026-11-19

In a technical debate about whether transformers have 'memory,' the author argues a prior is simply a vector that survives attention layers and next-token sampling. Transformers treat every token alike; distinctions between instructions, observations, state, examples and memory are human labels, not transformer primitives. A single forward pass doesn't become memory recall just because you name a behavior that way: 'A transformer is not a database or a brain. There's no memory, a self or cognition anywhere.' Human traits are projections, not technical reality.

Related event: Developer slams anthropomorphic AI terminology(3 posts)→

Original post →

More from Models

Models channel →