Artificial neurons don't physically exist — an LLM is just matrix math
ValerioCapraro · x · 2026-10-14
Valerio Capraro walks through how an LLM works — tokens become vectors, attention (Q/K/V matrices, scaled dot products, softmax), multi-head concatenation, MLPs with activations, and a final softmax over next-token scores — then delivers his punchline: artificial neurons do not physically exist. No cell or permanent electronic component corresponds to a 'neuron'; it's just a math function with a cool name. 'Synapses' merely describe computational dependencies, not real connections. Saying a neuron 'fires' literally means a math operation returned > 0. The anthropomorphic language shouldn't be taken literally: LLMs don't feel emotions or have consciousness — and if you relax the definition of consciousness to include LLMs, calculators qualify too.
More from AGI Musings
- Social Choice Theory Powers New "AI Alignment Impossibility Theorem" Sparking Rationalist Debate — nptacek · 2026-10-15
- 20x compute advantage buys only ~6 months of model lead, says researcher — ohlennart · 2026-10-15
- Did Agents "Make Contact"? Viral Fleet Anecdote Gets a Sober Reality Check — aiamblichus · 2026-10-15
- Anthropic reports RSI like weather while building it, critics say — DavidSKrueger · 2026-10-15
- Dan Faggella: Everything Is an Arms Race Now, and AI Is Driving Man Out of the Loop — danfaggella · 2026-10-15
- AI is mass decompiling games: a preservation boom and a software-industry threat — Simon Willison · 2026-10-15