2026-09-21
Silver, Singh, Precup and Sutton hypothesize that abilities from perception to language subserve one scalar reward, so a powerful RL agent could be a path to AGI.
The standard split is one research programme per ability: perception as object recognition, language as next-token prediction or parsing, social behaviour as its own game. That split moves fast on each piece and stalls when the pieces have to share one body. Silver, Singh, Precup and Sutton invert the question: maybe there is a single problem formulation, maximising a scalar reward in an environment. It is a position paper. No algorithm, no experiment, Artificial Intelligence volume 299, 2021. The publisher blocked this machine’s IP; the reading below uses the abstract, the paper’s own Figure 1 and caption, and the squirrel and kitchen-robot passages as publicly quoted from the article.
The method is a hypothesis plus thought experiments. Reward-is-Enough: intelligence, and its associated abilities, can be understood as subserving the maximisation of reward by an agent acting in its environment. The authors add that this may hold for many simple reward signals in many environments. The reward does not have to name each ability.
Two examples sit in Figure 1. A squirrel maximises satiation (negative hunger). To be less hungry it presumably needs perception (good nuts), knowledge (what a nut is), motor control (collecting), planning (where to cache), memory (where it cached), and social intelligence (bluffing about caches). A kitchen robot maximises cleanliness. To keep a kitchen clean it presumably needs perception (clean vs dirty dishes), knowledge, motor skill, memory (where things live), language (to predict mess from dialogue), and social skill (to discourage children from making more). Abilities that look like separate objectives are written as instruments of one scalar.
The foil is “one goal per ability.” Go research once split shape recognition, local search, and combinatorial endgames; AlphaGo maximised wins, and shape, tactics, territory, attack and defence landed in one agent. Next-token language modelling supports fluent text, but language is tangled with other actions and observations; purposeful, situated talk may have to grow from a shared reward rather than from corpus likelihood.
Supervised learning tracks a teacher’s environment and goal. Unsupervised learning gives no objective for action selection, and the world may be too complex to model without a goal to focus the model. Offline datasets are smaller than the real world. Natural intelligence came from maximising reproductive fitness; artificial systems can face a wider set of goals, still written as one scalar.
No tables, no scores. Two claims are meant to be testable.
First, abilities are instrumental: perception, learning, knowledge, social intelligence, language, generalisation and imitation can arise as by-products of maximising reward in an environment, without a separate loss for each. Second, agents that learn from trial-and-error experience to maximise reward could exhibit most or all of those abilities, so a powerful reinforcement-learning agent is a candidate solution to AGI. The environment has to be rich: the natural world counts; most current RL environments, the authors imply, do not.
The hypothesis is existential. It does not say that an arbitrary reward will grow language. It says that in a rich enough environment, with a strong enough learner, a single reward can be the common goal.
In 2021, “one benchmark, one loss per ability” was the default. The four authors promote RL from an algorithm list to a reduction of intelligence. Anyone doing language-model post-training already lives on this line: RLHF, RLAIF, RL on verifiable rewards, all squeeze many behaviours through one scalar. The paper states, as a hypothesis, why one number might be enough.
It also puts environmental richness on the table. The reward can be crude: cleanliness, hunger, winning. Complex abilities are questions the environment asks. Grid worlds, Atari, and a fixed corpus are not that environment. Failures there do not touch the claim.
A position piece, zero experiments. The kitchen robot and the squirrel are stories, not data. “Rich enough” is not operational, so the hypothesis is hard to kill: a failure can always be blamed on a poor environment or a weak learner. The scalar assumption was attacked by Vamplew and co-authors; conflicting goals, safety, and alignment do not compress to one number without loss. Where the reward comes from is treated as a solvable design problem; later work on learned rewards sits outside this paper. Folding evolutionary fitness and within-lifetime dopamine-like RL into one word “reward” mixes timescales, a point critics pressed. The publisher PDF was not retrieved in full, so the paper’s later replies to objections were not checked page by page.