Ajeya Cotra: redacted evidence undermines AI risk verification and safety cases
ajeya_cotra · x · 2026-11-06
AI safety researcher Ajeya Cotra laments that much of the underlying evidence in a recent report was redacted, arguing evidence transparency is essential for the world to assess loss-of-control risk in time.
Context: after a wave of misalignment incidents, OpenAI and Anthropic both reported slowing RL training for safety, fueling debate over pacing commitments and third-party auditors. Cotra argues we're putting the cart before the horse — the science of loss-of-control risk is nascent, companies make no standardized verifiable risk claims, alignment benchmarks are easily gamed by training, and even months-horizon risk can't be confidently bounded. Before verifying and auditing, she says, the evidence itself must be transparent.
More from AGI Musings
- Researcher: Labs disrespecting model interiority may undermine alignment — repligate · 2026-11-06
- When AI Can Make Anything, Do We Still Know What We Want? — jxnlco · 2026-11-06
- AI Practitioner Warns Progress Is Outpacing Society's Reaction to It — AndyMasley · 2026-11-06
- AI has drastically cheapened life's labor — gifts, parenting, friendships — in just 24 months — juliey4 · 2026-11-06
- Delvetown, a human-AI agent society, publishes a weekly newspaper written by humans and agents — Miles_Brundage · 2026-11-06
- Most people don't know what a thought is, says aphantasia thread observer — justalexoki · 2026-11-06