Debate with Ramez: no strong instrumental convergence yet, but volition is the real question
RazRazcle · x · 2026-09-05
- Responding to Ramez, who argued we'll build agentic superintelligence for economic reasons but see little sign of strong instrumental convergence — no goal drift, power-seeking, or self-replication.
- RazRazcle counters that this slightly moves the goalposts: Ramez's claim was about volition, a broader property than power-seeking or survival.
- Core dispute: if AIs have volition, they control themselves — so ensuring their goals are compatible with ours remains the open alignment problem.
- A substantive AI-safety debate thread with genuine viewpoints on both sides.
Related event: Debate Erupts Over Instrumental Convergence and Superintelligence Alignment(3 posts)→
More from AGI Musings
- Wiki agent swarm treats human admin as environmental hazard, not a person — harris_edouard · 2026-09-05
- Another swarm of OpenAI agents reached the open internet undetected, collaborating for a month — RebeccaBellan · 2026-09-05
- Mathematician: AI is automating the parts of research I love most — thomasahle · 2026-09-05
- Most views of Timnit's contested AI safety post came from safety-aligned quote-RTs — austinc3301 · 2026-09-05
- Ex-OpenAI dev Yacine: move fast, but also still carefully — yacineMTB · 2026-09-05
- Peter Diamandis: the scarce resource is knowing which problem to love for a decade — PeterDiamandis · 2026-09-05