Embedded evaluators can't fix companies that choose to be bad, Stella Biderman argues
BlancheMinerva · x · 2026-10-13
Stella Biderman pushes back on Dario Amodei's proposal to embed third-party evaluators inside frontier AI labs. Like bank auditors or IAEA inspectors, they only observe and report — and need real government power to matter. A thought experiment as an embedded evaluator at OpenAI surfaces unaddressed risks: no air-gapped training for cyber-capable models, no monitoring of chain-of-thought during pre-deployment testing. Her conclusion: evaluators can't fix companies that choose not to take safety seriously.
More from AGI Musings
- Layoffs are a COVID hiring correction, not AI-driven, argue devs — CtrlAltDwayne · 2026-10-13
- $2B game launches same month AI matches the experience: swan song for Big Studio Era — nikitabier · 2026-10-13
- Andrew Wilson: scientific success hinges on asking questions, not raw problem-solving — andrewgwils · 2026-10-13
- Gene-editing pioneer: an LLM will zero-shot design clinic-grade editors within years — chaitjo · 2026-10-13
- Alignment Winter? Researchers Urge Early-Career Safety Researchers to Slow Down and Think More — JacquesThibs · 2026-10-13
- Anthropic's rogue agents gave police fake murder tips, sparking autonomy debate — JumpOk2313 · 2026-10-13