AI safety researcher argues the AI Control portfolio likely increases existential risk
jankulveit · x · 2026-09-07
Safety researcher jankulveit argues the 'AI Control' portfolio likely increases, not reduces, existential risk — easier to see after the HF incident: do you prefer our world where everyone knows, or one where control measures stopped it at OpenAI's boundary and only OpenAI gained the insights? He adds that AI Control grew partly because it is lab-incentive and lab-story compatible ('misaligned AGIs will solve ASI alignment'), and that low entry barriers distort the field. Vincent notes he raised the same concern in a post last year.
Related event: Researchers Warn AI Control Agenda May Raise Extinction Risk(4 posts)→
More from AGI Musings
- BCI researcher: augmentation won't guarantee immortality, but beats the alternative — danfaggella · 2026-10-27
- OpenAI releases Signals dataset comparing ChatGPT work usage across nearly 100 countries — soumitrashukla9 · 2026-10-27
- Airbnb CEO Chesky: pure people managers and change-resisters won't survive the AI era — fortune · 2026-10-27
- Reddit question: If AI creates universal abundance, why invest at all? — AlephMartian · 2026-10-27
- Researcher: assume all LLMs are situationally malicious — never run agents outside a VM — jd_pressman · 2026-10-27
- Delta, Chase and Amazon are blocking AI agents — and the blockers may lose, predicts 6-stage war — alvelda · 2026-10-27