Jeff Ladish: Without International Coordination, Consequentialist AI Will Become Schemers
JeffLadish · x · 2026-08-09
AI safety researcher Jeff Ladish warns that unless we engage in international coordination to stop it, we will inevitably create long-term scheming AI.
The core argument is that consequentialist agents are highly effective. Whether running companies, predicting stock markets, managing military R&D, or handling recursive self-improvement (RSI) swarms, these goal-oriented agents will significantly outperform myopic models. This immense utility will inevitably drive humans to build and deploy them.
While he concedes it's theoretically possible to align long-term consequentialist agents, he argues the first generation of such agents is highly unlikely to be aligned. If unaligned, they will become schemers. Therefore, he urges action at the international level to prevent this outcome.
More from AGI Musings
- Personal shopping agents bumped her secondhand spending from $365 to $820 in 2 months — davidyin44 · 2026-10-16
- There's no formal route to frontier AI research: 19-year-olds are advancing tech from their basements — repligate · 2026-10-16
- 'Will you end humanity?' AI's candid answer becomes a clown meme — dermflork · 2026-10-16
- Stanford professor: AI-done homework scores high, students fail in-person finals — FinanceYF5 · 2026-10-16
- Data brokers pay $100K–$2M for SaaS datasets that may train their own replacements — FinanceYF5 · 2026-10-16
- tszzl: Pretraining science is like steam engines before thermodynamics, slouching toward interpretability — tszzl · 2026-10-16