RL Scholar Szepesvári Argues Real AI Risk Lies in State Rivalry, Not Rogue Agents

On September 14, Csaba Szepesvári, a leading scholar in reinforcement learning, posted a systematic Twitter thread on how to think about AI risk. His core conclusion: a single highly intelligent agent does not pose an existential risk, but macro-level structural risks deserve attention.

Confirmed

• Szepesvári proposed a mental model: replace 'AI' with 'a smart person' or 'a powerful organization' when reasoning, since human intelligence is the only general-purpose intelligence technology humanity has long experience living with, and over centuries we have built guardrails around it—education, institutions, laws, professional norms, and access controls. This framework, relayed by @AdaptiveAgents, was endorsed as a workable starting point for thinking about AI risk.
• In step-by-step reasoning about extinction scenarios, he argued that even if a power-holder deliberately sought human extinction, the actual probability of achieving it remains very low—enormous harm is possible, but extinction is unlikely; accidental extinction is even less likely, though not zero.
• Swapping the human power-holder for an AI, he believes the extinction probability would not change.
• He noted he sees plenty of risks, but most are not existential, and in this scenario humans still remain in charge.

Why it matters

• Szepesvári pointed out that the real risk lies in larger structures: the jockeying among powerful organizations such as nation-states and their feedback loops—risks are dramatically amplified if these organizations are further equipped with armies of 'cheap, obedient, superhuman-level' agents.
• As an authoritative figure in reinforcement learning, his analytical framework offers the public a demystified lens for rationally assessing AI existential risk, shifting the discussion from 'a lone AI rebellion' toward governance and international competition.

2026-09-14 ~ 2026-09-14 · 6 related posts

• Treating AI Like a Capable Human: A Framework for AI Risk and Accountability — AdaptiveAgents · 2026-09-14
• [source] RL veteran Szepesvári models AI risk by swapping 'AI' for 'a human' — CsabaSzepesvari · 2026-09-14
• Szepesvári, part 2: many AI risks, but mostly non-existential — CsabaSzepesvari · 2026-09-14
• [source] RL Pioneer Szepesvari: A Single Human In Charge Of AI Poses Low Extinction Risk — CsabaSzepesvari · 2026-09-14
• [source] Szepesvari: Real AI Extinction Risk Lies In State Competition With Agent Armies — CsabaSzepesvari · 2026-09-14
• Csaba Szepesvári: worst AI scenario is short-sighted nation-state rivalry — CsabaSzepesvari · 2026-09-14