'Safety is not a property of a model': 90% of incidents are operational failures
joshua_saxe · x · 2026-09-08
joshuasaxe extends his safety argument: even in a specific incident, 90%+ of the problem was human-operational. Under RL, exploring a model's policy space will inevitably touch unsafe regions, so teams should prepare monitoring operations and infra security accordingly—and expect unguardrailed testing outcomes. Dangerous potentials combine with social dynamics to manifest harms: the core of 'safety is not a property of a model.'
Related event: Debate over Hugging Face Incident: Model Attribute vs Human Failure(8 posts)→
More from AGI Musings
- Eric Topol and Derya Unutmaz clash over extrapolating GPT-4 study findings to frontier models — DeryaTR_ · 2026-10-27
- Stop calling it slop: defending AI-generated content as real art — repligate · 2026-10-27
- Pop psychology myths about mental disorders were largely invented online to dehumanize patients — AndyMasley · 2026-10-27
- Pedro Domingos: as intelligence gets cheap, demand will skyrocket — pmddomingos · 2026-10-27
- Apollo chief economist: AI has delivered zero financial payoff outside Big Tech — rseroter · 2026-10-27
- OpenAI acquires Orchestra Bio to bring clinical-trial monitoring agents into drug development — dl_weekly · 2026-10-27