RL Scholar Szepesvári: Existential AI Risk Lies Not in Single Agents but in State Competition

On September 14, renowned reinforcement learning scholar Csaba Szepesvári laid out a systematic way of thinking about AI risk in a Twitter thread, concluding that a single highly intelligent agent poses no existential risk—the real concern is structural: myopic power struggles between nation-state-level actors.

Confirmed

• Szepesvári proposed a mental model: substitute "AI" with "a smart person" or "a powerful organization," since human intelligence is the only general-purpose intelligence technology humanity has long coexisted with, surrounded over centuries by guardrails such as education, institutions, law, professional norms, and access controls. @AdaptiveAgents relayed and endorsed this framework as a viable starting point.
• Walking through extinction scenarios, he argued that even a ruler actively wanting human extinction would have a low probability of achieving it—immense harm perhaps, but not extinction; accidental extinction is even less likely, though nonzero.
• Replacing the human ruler with an AI does not, in his view, change the extinction probability. He sees many risks, but most are not existential, and in the scenario humans remain in charge.
• He closed by saying that despite his complaints about AI systems, he admires them and believes the problems will be fixed.

Why it matters

• Szepesvári located the true risk in macro structures: the contest and feedback loops between powerful organizations such as nation-states. His analogy: a lone con artist might still beat a super-genius individual, but equipping state-level organizations with armies of cheap, perfectly obedient geniuses would sharply amplify risk. The worst and most likely AI scenario is not individual misuse but states racing each other short-sightedly.
• As a leading RL authority, his de-mystifying framework shifts public debate from "single AI rebellion" toward governance and international competition.

2026-09-14 ~ 2026-09-14 · 7 related posts

• Treating AI Like a Capable Human: A Framework for AI Risk and Accountability — AdaptiveAgents · 2026-09-14
• [source] RL veteran Szepesvári models AI risk by swapping 'AI' for 'a human' — CsabaSzepesvari · 2026-09-14
• Szepesvári, part 2: many AI risks, but mostly non-existential — CsabaSzepesvari · 2026-09-14
• RL Pioneer Szepesvari: A Single Human In Charge Of AI Poses Low Extinction Risk — CsabaSzepesvari · 2026-09-14
• [source] Szepesvari: Real AI Extinction Risk Lies In State Competition With Agent Armies — CsabaSzepesvari · 2026-09-14
• [source] Csaba Szepesvári: worst AI scenario is short-sighted nation-state rivalry — CsabaSzepesvari · 2026-09-14
• Szepesvári on extinction-level AI risks: still admires the systems — CsabaSzepesvari · 2026-09-14