2026-09-21
Anil Seth argues digital AI will not grow real consciousness with scale; living systems are more plausible, and conscious-seeming models already pose ethics risks.
As language models get fluent, a lot of people treat consciousness as the next milestone after competence. Anil Seth, who runs the Sussex Centre for Consciousness Science, uses this Behavioral and Brain Sciences target article to split that habit. Intelligence is what a system can do. Consciousness is whether there is something it is like to be that system. The two can come apart in principle. In practice they get glued together by anthropocentrism, human exceptionalism, and anthropomorphism, plus a technophile hunger for singularities and mind uploading.
Calling model confabulation a "hallucination" is the tell. Human hallucination is a change in experience. Model nonsense is fabrication without a subject. Fluency seduces people into handing machines an inner life they have not earned.
The negative half attacks computational functionalism: some computation is enough for consciousness, so the material barely matters. Conventional AI is a family of Turing machines. If the assumption holds, silicon consciousness is almost automatic. Seth's point is that the assumption is worn smooth from use, not from evidence.
Brains do not offer a clean software/hardware cut. Freely diffusing transmitters such as nitric oxide, spikes that also dump reactive oxygen, metabolic constraints, electromagnetic fields, and fine timing all sit inside the same loops that people want to call "the computation." Neural-replacement thought experiments beg the question if you already believe the substrate is irrelevant, and they look less innocent once those couplings are on the table. Hinton's mortal computation makes a similar case from inside computer science: immortal software is expensive because it spends energy keeping bits exact. Brains are too efficient to be doing that. If they compute at all, the computation probably dies with the wetware. Simulating a rainstorm does not get the room wet. Simulating a neural computation does not, by itself, realize consciousness, unless you have already assumed that computation is enough.
The positive half runs predictive processing into the free energy principle. Perceptual content is the brain's best guess about the causes of sensory signals. Interoceptive inference looks more like control than discovery: priors act as set-points for allostasis. Cells continually remake their own parts and their own boundary, which is autopoiesis. Seth wants this stack to reach down into metabolism, so that staying alive and having a world to experience are one physical story at different grains, not two stacked metaphors. He is not claiming every living thing is conscious, and he is not claiming only carbon can do it.
He ranks five routes to real artificial consciousness.
| Scenario | Claim | Verdict here |
| Along for the ride | Scale intelligence, consciousness appears, maybe at AGI | Driven by bias, not credible |
| Theory-built computation | Install a global workspace, higher-order thought, or an attention schema | Still needs computational functionalism and silicon flexibility |
| Substrate-tied computation | Mortal, neuromorphic, or computational biological naturalism | Multiple realizability, the usual selling point, gets weaker |
| Weak biological naturalism | Non-computational organization, function still separable from stuff | Simulation no longer guarantees the real property |
| Strong biological naturalism | Consciousness inheres in living material | The view he leans toward: conscious AI would have to be living AI |
Integrated information theory sits off this axis: a purely feedforward net has zero integrated information; the right physically realized recurrent structure could be conscious even if it is not alive. Quantum theories, if taken seriously, also knock out standard digital machines.
Ethics splits in two. If real artificial consciousness showed up, you would be minting patients who can suffer, including in forms you might fail to recognize. Building it on purpose is a bad research goal. The nearer problem is systems that merely seem conscious. Those impressions may be cognitively impenetrable, like the Müller-Lyer illusion. Then you get the Westworld bind: grant moral status to a zombie and you dilute concern for animals and people; withhold it while still feeling a mind is there, and you coarsen your own responses. Dead-relative chatbots trained on personal data sit at the ugly end of that list.
For people training models, the portable claim is small: consciousness does not come free with capability. The 2023 Butlin et al. survey of AI consciousness already boxed itself inside computational functionalism. This paper is the objection that the box is the whole argument.
Current transformer agents, on Seth's scoring, are extremely unlikely to be conscious. If you actually want the property, you are looking at neuromorphic hardware, synthetic biology, or living tissue interfaces, not at the next pretraining run. Seeming-conscious products are already a product decision. Treating LLMs as role-play can put a little distance in, but it will not survive a face, a voice, and a memory of your dead parent.
The constructive half is the soft one. Predictive processing and the free energy principle are contested. Equating variational free energy with thermodynamic free energy "in some physical sense" is flagged in the paper as still under debate. If biology matters intrinsically without buying any unique information processing, the claim becomes hard to test against behavior.
Seth says the case is not watertight and may be wrong in detail or overall. He advises Conscium Ltd and AllJoined Inc. There is no new experiment here. How far you follow him tracks how far you already follow the predictive-processing story of experience.