Loss of control is an open science problem — auditors shouldn't be billed as a safety guarantee
ajeya_cotra · x · 2026-09-27
Ajeya Cotra argues preventing loss of control remains an open scientific problem: science requires public evidence, so companies and third-party evaluators must publish far more concrete risk data before safety standards can be agreed. Nat Purser adds that independent audits can't honestly be billed as a "safety guarantee" — oversight is an essential but unglamorous part of governance that can spur hearings, legislation, and industry-wide rules while democratizing critical decisions.
More from AGI Musings
- Safety testing could be AI's third compute dimension, author argues NVIDIA is pushing regulatory capture — JOBhakdi · 2026-09-27
- Claiming AI-fatigue today is like claiming web-fatigue in the early 2000s — pattyneta · 2026-09-27
- Nate Silver: AI is a big win for 40-somethings who learned the right way — soumitrashukla9 · 2026-09-27
- 'Agentic AI Killed DevOps' Claim Sparks Developer Pushback — emeka_boris · 2026-09-27
- Who verifies AI-written code? Human evals may become the bottleneck — xx_HeckinChonker_xx · 2026-09-27
- Gary Marcus: Everyone would freak out if Chinese AI did what OpenAI is doing — GaryMarcus · 2026-09-27