The 'Stochastic Parrots' Debate Reignites Over What LLMs Really Are

Episode 1 · Researchers Debate Whether "LLMs Are Just Next-Token Predictors" Is a Meaningless Claim (2026-09-25, 7 posts)

On September 25–26, multiple researchers debated the popular claim that "LLMs are just next-token predictors / stochastic parrots" on Twitter/Bluesky. Aaroth kicked it off, arguing the statement itself is contentless—any mapping from input strings to output strings can be decomposed into a sequence of next-token predictions, so using "it does next-word prediction" to belittle or define LLMs conveys no substantive content.

Confirmed

• Aaroth posted this view on September 25 and revisited history: ten years ago, "whether representing the output distribution as next-token distributions poses computational difficulties that prevent models from learning useful behaviors" was a genuinely interesting question—but practice showed the concern never materialized; the next-token representation created no hard learning problem.
• Princeton researcher Aran Nayebi relayed the debate on September 26; participants included MIT's Andrew Ilyas and Aaron Roth, with Aaron Roth likewise noting that "next-word prediction" as a framing lacks content.
• Another thread of the debate was Aaroth's clash with @10001plate over how the human brain works. @10001plate countered that saying "the brain picks the next word by producing machine-learning scores" is flat-out wrong: first, the brain can think out of order; second, it has a mechanism for choosing "what to say" different from an LLM's.
• Aaroth responded further: any probability distribution can mathematically be represented as a product of per-element conditional distributions (exactly an LLM's output form), but that doesn't mean the system producing it is inherently limited to sequential reasoning; arguing an LLM can't think beyond sequence because "it uses a particular representation" doesn't hold.

Why it matters

• The debate touches on a common misreading of LLM capability boundaries: conflating "output form" (token-by-token generation) with "internal mechanism or reasoning ability" (can only think sequentially)—a confusion made both when dismissing and when deifying LLMs on social media.
• It also ties together two threads: the decade-old question about the learnability of next-token representations has been answered by practice, while whether the human brain can itself be viewed as some kind of "prediction machine" remains open.

• 'Next-token predictor' is a contentless way to describe LLMs — Aaroth · 2026-09-25
• The decade-old worry that next-token representation would hinder learning turned out unfounded — Aaroth · 2026-09-25
• Why 'LLMs are just next-token predictors' is a contentless argument — Aaroth · 2026-09-25
• Researchers call 'next token predictor' a contentless critique of LLMs — aran_nayebi · 2026-09-26
• "Brains aren't next-token predictors": the counterpoint in an LLM-vs-brain debate — 10001plateaus · 2026-09-26
• Debate on X: Is a brain a next-token predictor? It's just math, argues one side — Aaroth · 2026-09-26
• LLM output distributions being factored doesn't mean they can't think out of order — Aaroth · 2026-09-26

Episode 2 · Melanie Mitchell Sparks Fresh Debate Over "Stochastic Parrots" and What Counts as an LLM (2026-09-27, 28 posts)

On September 27, Santa Fe Institute professor Melanie Mitchell posted on X that "what we have now are not LLMs," sparking a chain of heated exchanges among researchers over the Stochastic Parrots paper and the definition of "LLM." Mitchell's core argument: a language model, by its original meaning, is a system that captures the statistical structure of language (n-gram models, Elman networks, and GPT-2 all qualify), whereas today's AI systems have undergone extensive post-training and incorporate numerous external software components, putting them beyond that scope; therefore, using "stochastic parrots" to criticize today's systems is a straw man—the paper was aimed at the LLMs of 2021.

Confirmed

• Mitchell and Boaz Barak (University of Chicago) clashed over definitions: Barak pressed on whether "LLM" and "stochastic parrot" only apply to pre-InstructGPT models, noting that people were always going to keep calling today's systems LLMs; Mitchell responded with a long post reiterating her understanding of the "language model" lineage.
• Google DeepMind researcher Andrew Lampinen posted a lengthy thread systematically rebutting the core technical claims of the Bender & Koller paper, arguing that systems trained on language alone can still learn meaning, and calling the "stochastic parrot" phrase itself "much noise, signifying nothing."
• Blanche Minerva of Redwood Research laid out a five-step pipeline (pretraining, teaching tool-calling tokens, teaching the command line, etc.) to counter the claim that "LLMs are just next-token predictors," and asked Mitchell: is a transformer with command-line access that can run programs a stochastic parrot, and does the verdict depend on how it was trained? mmitchellai responded that he and Emily Bender had written a piece explaining how "stochastic parrot" refers to LLMs.
• Timnit Gebru, one of the paper's authors, tweeted that she has no interest in engaging with people who keep harping on the argument, asking "why does an accurate description of LLMs upset you so much"; cloneofsimo quote-tweeted with an image in response.
• Other voices backing Mitchell: Brad Buchsbaum said the paper "aviarized" LLMs; tallinzen, unusually siding with the paper, argued that using today's systems to attack it is anachronistic. @aidanclark separately mocked current models as far from ASI using a botched imagegen output.

Unconfirmed

• No consensus on the boundaries of "LLM": whether systems after RLHF and tool use still count as language models, and whether the "stochastic parrot" critique is thereby invalidated, remain open disagreements with no settled conclusion.

Why it matters

• The dispute touches on whether one of the most-cited critical frameworks in AI still applies: if today's systems differ in constitution and capability from the LLMs of 2021, much of the safety and ethics discourse around "stochastic parrots" may need recalibrated terminology and targets.

• Researcher pushes back on 'stochastic parrot' framing: even pure LLMs no longer fit it — BlancheMinerva · 2026-09-27
• Mitchell points to his and Emily Bender's essays on how "stochastic parrot" refers to LLMs — mmitchell_ai · 2026-09-27
• Melanie Mitchell: 'Stochastic parrots' is a strawman against RL-post-trained models — MelMitchell1 · 2026-09-27
• Blanche Minerva's five-step pipeline shows why "just next-token prediction" no longer fits LLMs — BlancheMinerva · 2026-09-27
• Researcher asks: a transformer with a command line — still a 'stochastic parrot'? — BlancheMinerva · 2026-09-27
• Dev challenges "stochastic parrot" critics: a transformer with a shell — how many lines until it's one? — BlancheMinerva · 2026-09-27
• A meme answers Timnit Gebru's 'stochastic parrots' defiance — cloneofsimo · 2026-09-27
• Mitchell: stochastic parrot critique targeted 2021 LLMs, not today's AI systems — PMinervini · 2026-09-27
• Mel Mitchell argues today's AI systems are no longer LLMs, term misuse muddies debate — _aidan_clark_ · 2026-09-27
• User roasts imagegen being 'dumb as bricks' as evidence we're nowhere near ASI — _aidan_clark_ · 2026-09-27
• Melanie Mitchell backs claim that today's systems aren't really LLMs, citing original definition — MelMitchell1 · 2026-09-27
• Melanie Mitchell vs DeepMind Co-founder: Do Today's Models Still Count as 'Stochastic Parrots'? — _aidan_clark_ · 2026-09-27
• Boaz Barak pushes back on Mitchell: were 'stochastic parrots' only pre-2022 models? — boazbaraktcs · 2026-09-27
• Melanie Mitchell vs. Boaz Barak: does 'stochastic parrots' still apply to today's LLMs? — MelMitchell1 · 2026-09-27
• Melanie Mitchell: language models span n-grams to GPT-2, agentic systems are another matter — MelMitchell1 · 2026-09-27
• Melanie Mitchell argues RLHF-trained models are no longer true language models — MelMitchell1 · 2026-09-27
• Melanie Mitchell: Post-Training Like RLHF Makes Models No Longer Statistical Language Models — MelMitchell1 · 2026-09-27
• Debate: raw LLMs without any harness already refute the 'stochastic parrot' claim — Zergylord · 2026-09-27
• Ex-Google Research VP Aidan Clark: "What We Have Now Are Not LLMs" — giffmana · 2026-09-27
• Google researcher Lampinen pens long thread rebutting the stochastic parrots argument on LLM meaning — AndrewLampinen · 2026-09-27

8 more related posts →