Floridi: token-completion LLMs mimic abduction without a check

What Kind of Reasoning (if any) is an LLM actually doing? On the Stochastic Nature and Abductive Appearance of Large Language Models

Luciano Floridi, Jessica Morley, Claudio Novelli, David Watson

cs.CL, cs.AI

2025-12-11

Floridi argues that token-completion LLMs only sample likely text. They look abductive because training text is full of human explanations, yet they cannot verify hypotheses.

What problem this solves

Ask why a car will not start in the cold and a chatbot lists a weak battery and thick oil, then calls the battery most likely. That shape is abduction: from an effect, guess a cause that new evidence could defeat. Peirce named the move. Harman added the comparison, inference to the best explanation (IBE): among hypotheses, pick the one that would account for the evidence best if it were true.

Pareschi (2023) reported that GPT-4 could attach plausible stories to symptoms and crime-scene clues. On αNLI, where the task is to choose which of two endings explains a story, performance is often described as near-human (Balepur et al., 2024). Floridi, Morley, Novelli, and Watson ask what computation that performance is. They stay with mainstream text-only token-completion models, the GPT series as the reference, and set multimodal models and other generative designs aside.

Method

There is no new model and no new benchmark. The split is between how text is produced and why it reads like an explanation.

Weak abduction only offers a story that could be true. Strong abduction then compares candidates on simplicity, fit with background knowledge, and coverage. That second step is IBE. On an open prompt a model often emits several hypotheses. When the options are written out, it picks the ending a person would be more likely to choose. αNLI tests that second step (Bhagavatula et al., 2020).

Training is still next-token prediction. A Transformer uses causal self-attention to fit the distribution of the next token given the prefix. The weights store co-occurrence and sentence habits. Generation is greedy or samples with a temperature, the knob for how random the draw is. Shanahan (2022) uses a Moon-landing prompt to fix the point: the answer is the continuation most common for that prefix in the training text, not a lookup in a knowledge table.

In statistical terms this is prior predictive sampling. The model draws from a distribution it already learned. Nothing outside the text updates a posterior. Reichenbach's context of discovery, where hypotheses are proposed, is what the model imitates. The context of justification, where hypotheses are checked, is missing unless retrieval, a code interpreter, or a person is bolted on later.

The explanatory tone comes from the corpus. Human writing is full of the word because and of question-answer templates. Train only on astrology and the answers come out astrologically plausible. Step-by-step derivations are templates too. Prompting the model to think step by step pushes it into phrasing that co-occurs with correct solutions (Wei et al., 2022; Kojima et al., 2022). Co-occurrence is not deduction starting up. Following Kojima and colleagues, the paper calls the behavior zeroth-order abduction: given a prompt, continue with text that looks like a hypothesis.

In a dialogue written for the paper, spilled coffee and a person on the kitchen floor produce both a slip and a faint, with a lean toward the slip. On rarer problems the same habit invents syndromes and citations, because a complete, confident answer looks more like the training text than an admission of ignorance. Continuations the user wants to hear are preferred in the same way. The paper calls the refusal to stay silent over-abduction, and counts sycophancy (Sharma et al., 2023) in that class.

Results

The paper reports no measurements of its own.

SetupComparisonWhat this paper does with it
Mainstream LLMs on αNLIHumansCited as near-human (Balepur et al., 2024); no gap given
GPT-3/GPT-4 on analogies and Raven-style itemsHumansWebb et al. (2023) report parity or better on some items; treated here as template familiarity, brittle if the item changes
GPT-4 and others on SCARStructural alignment of analogiesYuan et al. (2023): familiar analogies are produced; new ones lose the relation
GPT-3.5 and GPT-4Acceptance of fallaciesPayandeh et al. (2024): often accepted unless asked to critique; no rate given
Human-model hypothesis listsWhether a person can follow them upZhou et al. (2024) and Pareschi (2023): some ideas are worth checking, mixed with irrelevant ones

Sampling is the core. Abduction is the surface. Occasional wins over people are read as overfitting to common templates: many solution styles sit on top of each other in the data, and a changed wording exposes the gap. How large that gap is, the paper does not measure.

Five replies tighten the scope. The analogy covers output structure; the model has no beliefs. Beating humans does not show genuine reasoning. Causal common sense is co-occurrence, so the word smoke is followed by fire, with no bodily check. The account is limited to large models and suitable prompts. Later GPT models are credited mainly to scale and data. The remark that GPT-5 checks more points comes with no table. GPT 5.1, at the time of writing, is still described as token completion, and its reasoning mode is a hidden draft of tokens. Accurate arithmetic is offloaded to a calculator. Real abduction, if it arrives, would have to be added outside the engine as well.

Why it matters

A leaderboard that rewards explanation-shaped answers is scoring sentence probability. A drop after the wording changes is what this account predicts.

The usable slot is discovery. Experimental anomalies, case facts, and differential diagnosis can all be sent over for extra hypotheses. In the crime example from Pareschi, some hypotheses were worth chasing even though they were wrong. Checking belongs to a person, or to retrieval, a knowledge graph, or a verifier. Bare output is compared to an anonymous forum post. In medicine and law, adopting it treats the tone as if the evidence had already been weighed.

There is no new module here and no performance table. The paper accepts that these models cannot leave their corpus and cannot tell that they are wrong. It treats a convincing answer as systematic, because explanatory phrasing is regular in the training text and the interface presents the reply as reasoning.

Limitations

Section 7 states the authors' own limits. The sample is the large models of around 2025; smaller models show a weaker illusion. Future systems are left open. Memory and attention are not analyzed. Explanation is used in the ordinary sense, not under a formal test of unification or causal mechanism. Ethics gets one remark: a convincing false explanation invites people to treat a guess as knowledge, which is especially risky in medicine and law (Lenat and Marcus, 2023).

Almost no numbers travel with the citations. None of the named studies contribute a score, a sample size, or a baseline in the text, so near-human cannot be turned into a gap. The two dialogues were written for the paper. Remarks about GPT-5 and GPT 5.1 are not measured, and they are not compared with systems that search or verify.

The label zeroth-order abduction works against the argument. The claim is that no abduction takes place, while the label sorts outputs with abductive vocabulary. A footnote to Yuan (2023) is stricter: prompting solves a downstream task if and only if the task sits in the category induced by pretraining; otherwise prompting is provably insufficient. The body never turns that criterion into an evaluation procedure.

Terms

Source

What people are saying

Related papers

All paper explainers