Debate Over AI Risk Priorities: Extinction Risk vs Present-Day Harms

On October 1, tech writer Timothy Lee (@binarybits) and NathanpmYoung engaged in a multi-round debate on "AI risk priorities" on social media. The core disagreement: which deserves more attention now — long-term extinction-level risks or present-day harms.

Confirmed

• Lee openly acknowledged that some AI critics have indeed been too dismissive of various risks in the past; he noted that back in 2023 he wrote predicting the emergence of powerful AI hacking tools (even runaway AI), arguing we should try to anticipate future emergent risks and prepare in advance.
• Responding to Nathan's charge of "safety priority drift," Lee argued it's only natural for risk concerns to shift over time: three years ago models lacked the capability to "hack-like manipulate humans," so not over-focusing on such risks then was reasonable.
• Lee also said the takeover risk of "models trying to take over the world," while real, remains far from reality and doesn't warrant action for now; legislating preemptively for harms that may only materialize years later is impractical.
• Nathan's rebuttal centered on two points: first, the list of risk narratives keeps changing — the near-term harm ignored back then was algorithmic bias, yet today's x-risk narratives have pushed those issues out of focus; second, critics have long accused those focused on endgame risks of diverting attention from current risks, but he believes extinction-level risks deserve to be taken seriously, because mainstream consensus does not consider them unthinkable, and some risks once mocked as "thinking too far ahead" have already become present-day realities.

Why it matters

The debate reflects the long-standing "long-tail camp vs. present-day camp" tension within the AI safety community: as model capabilities rapidly advance, distant risks once dismissed as alarmist are edging closer to reality, while there is still no consensus on how resources and attention should be allocated, or whether legislation should come early. The two sides offer readers representative polar positions on the question of timescales in AI risk governance.

2026-10-01 ~ 2026-10-01 · 8 related posts

• [source] AI safety priorities shift too fast, critic argues as x-risk debate flares on X — NathanpmYoung · 2026-10-01
• Timothy Lee pushes back: AI risk focus should evolve as model capabilities do — binarybits · 2026-10-01
• [source] Timothy Lee: AI takeover risk is real but we're nowhere close, risks should track capabilities — binarybits · 2026-10-01
• AI risk debate: are extinction-level 'endgame' risks worth worrying about now? — NathanpmYoung · 2026-10-01
• AI Risk Debate: Once-Mocked 'Galaxy-Brained' Extinction Risks Are Becoming Today's Risks — binarybits · 2026-10-01
• [source] binarybits: We Should Forecast Emerging AI Risks, Warned of AI Hacking Tools Back in 2023 — binarybits · 2026-10-01
• Timothy Lee: legislating against years-away AI harms was never realistic — binarybits · 2026-10-01
• AI policy debate: liability rules over preemptive bans for future model risks — NathanpmYoung · 2026-10-01