Google researcher Lampinen pens long thread rebutting the stochastic parrots argument on LLM meaning

AndrewLampinen · x · 2026-09-27

Google DeepMind researcher Andrew Lampinen published a long thread systematically rebutting the core technical claim from the stochastic parrots paper (Bender & Koller): that systems trained on language alone can only learn form, not meaning.

Key points:

The generalizations that LMs trained from scratch in controlled settings exhibit — without tools or harnesses — already disprove the a priori impossibility argument; tool calls are just tokens like any other language, so "harnesses magically providing meaning" doesn't hold.
He argues claims that meaning can't be learned from language "prove too much," and discusses the memorization–generalization relationship in LMs and other ML systems.
He cites his own work on whether LMs can learn causal structure and reasoning from passive offline training, unpacking two conflated dimensions behind the term "passive."

His core frustration: the paper's authors never define terms, make falsifiable claims, or engage with contradicting empirical evidence — "it's fine to make a poor argument with flashy rhetoric, but you should update as evidence changes. And the authors clearly don't."

Original post →

Google researcher Lampinen pens long thread rebutting the stochastic parrots argument on LLM meaning

More from Models