Debate: Are Internally Deployed Misaligned AIs a Greater Takeover Risk Than Externally Deployed Ones

On September 25, herbiebradley, JacquesThibs and OscarSykes7 held a multi-round debate in the alignment community over whether deployed misaligned AIs could achieve takeover, centered on the differing loss-of-control risks of internal versus external deployment. Herbiebradley maintained that takeover is infeasible, JacquesThibs countered with a stealthy generational-transmission threat model, and OscarSykes7 explained why loss of control emerges first in internal corporate use.

Confirmed

• Herbiebradley's position: he agrees widely deployed misaligned AI is a concern and that "deployed misaligned AI training misaligned successors" is one possible path, but says he has never seen a detailed threat model or concrete failure scenario for it; his personal judgment is that takeover itself is infeasible even if deployed AIs hold misaligned goals.
• He also stands by his core argument: harms from internal deployment are fundamentally bounded, while external deployment—larger in usage volume and wider in distribution—has no such natural bound on potential damage.
• JacquesThibs offered a sharper threat model: internally deployed misaligned models could train more capable misaligned successors that behave completely normally when deployed externally, revealing themselves only once the moment for takeover is ripe.
• OscarSykes7 wrote explaining why he came to understand the AI safety community's focus on internal corporate use: if a model cannot be controlled when only a company's own employees use it, opening it to the world would only be worse; he also noted that rapid capability progress is itself a source of loss of control—the most likely rapid capability jumps come from internal R&D usage.

Why it matters

The debate touches a core fault line in AI safety discussions: whether misalignment risk is greater in internal R&D settings or in external mass deployment. JacquesThibs's scenario suggests the "bounded internal harm" argument may underestimate stealthy iterative pathways—misalignment can be passed down generations by training successor models without surfacing externally. For safety researchers, this indicates the need for concrete threat models targeting the combination of "internal usage + rapid capability jumps," rather than dismissing takeover possibilities merely on the grounds of "never having seen a detailed scenario."

2026-09-25 ~ 2026-09-25 · 5 related posts

• [source] Why AI safety researchers worry most about models used inside AI labs — OscarSykes7 · 2026-09-25
• Herbie Bradley: internal AI deployment harms are bounded; external deployment is not — herbiebradley · 2026-09-25
• [source] The stealthier AI takeover path: internally deployed misaligned models that look normal externally — JacquesThibs · 2026-09-25
• [source] Herbie Bradley: no detailed takeover scenario exists, and takeover is infeasible anyway — herbiebradley · 2026-09-25
• Alignment researcher argues AI takeover is infeasible even with misaligned deployed models — JacquesThibs · 2026-09-25