Researchers Debate Whether Current Alignment Methods Scale to Superintelligence

Around the claim that "we don't know how to align superintelligence," several AI researchers debated publicly. Alignment researcher Hugh Zhang stated on September 10 that methods used to align models today may not scale as models approach superintelligence; he acknowledged the field is deeply divided and said he would be "very happy" to be proven wrong.

Confirmed

• Hugh Zhang explicitly warned current alignment methods may not scale to superintelligence, while remaining open to falsification (m3, m4, m5).
• Imbue co-founder Josh Albrecht pushed back: no one truly believes a formal system can "perfectly" align an agent; absent perfection, alignment is an empirical, risk-per-effort question (m2, m5).
• Researcher @gandamuml raised a key point of divergence: what truly needs hardening is everything outside ML, since aligning one model does not align all models, and the barrier to training or retraining models keeps falling (m1).

Unconfirmed

• @gandamuml also suggested a counterintuitive view: so far, releasing everything as fast as possible without controls and letting everyone witness negative consequences may have been the better path—a personal opinion without full argumentation in the source material (m1).

Why it matters

• The debate touches the core controversy of AI safety: whether alignment is an engineering problem improvable through empirical iteration, or a fundamental challenge that fails as capability rises. Zhang's "only one chance to get it right" versus Albrecht's "risk-per-effort" framing captures two typical stances within the safety community, while the point about falling retraining barriers extends the problem from a single model to the whole ecosystem, with some framing alignment as a structural incentive problem rather than a technical one (m5).

2026-09-10 ~ 2026-09-11 · 5 related posts

• [source] Imbue co-founder: alignment isn't a perfect-formal-system problem, it's risk-vs-investment — joshalbrecht · 2026-09-10
• Alignment researcher fears current methods won't scale to superintelligence — hughbzhang · 2026-09-10
• [source] Alignment researcher warns current methods may not scale to superintelligence — hughbzhang · 2026-09-10
• Alignment is a structural incentive problem, not a technical one, argues Josh Albrecht — joshalbrecht · 2026-09-10
• [source] Researcher: Aligning One Model Doesn't Align All, Fast Uncontrolled Release May Have Been Better — gandamu_ml · 2026-09-11