Researcher says OpenAI's new third-party safety audits are a step back from METR access
DKokotajlo · x · 2026-11-17
OpenAI announced embedded assessments with Apollo Research, METR, Transluce and Goodfire, giving outside experts deep access to internal models to probe alignment and safeguards.
Researcher tomekkorbak argues the new engagements are a step back from OpenAI's earlier METR work:
- METR previously had access to OpenAI's monorepo during monitor red-teaming; that's now off the table.
- New engagements are narrowly scoped (a single RL run for METR, a single threat model for Apollo).
- Auditors only talk to "authorized" employees who receive OpenAI's "guidance" beforehand.
He says this makes it easy for OpenAI to cherry-pick models, data subsets and employees to paint a rosy risk picture; embedded auditing only works with employee-level access, not company-scoped contracting.
Related event: Researchers Say OpenAI's Embedded Safety Evaluations Fall Short(2 posts)→
More from Safety
- Anthropic quietly pitches its own AI safety bill to lawmakers — marigo · 2026-11-17
- Cambridge's David Krueger: AI safety community is too distracted by daily news horse races — DavidSKrueger · 2026-11-17
- Kelsey Tuoc: AI Companies Won't Give Embedded Auditors Real Employee-Level Access — connoraxiotes · 2026-11-17
- DEF CON talk breaks down 6 data-exfiltration vulnerabilities in AI agents — Kyrannio · 2026-11-17
- Frontier AI should be credibly destroyable or publishable by n-of-k actors — DavidSKrueger · 2026-11-17
- Polymarket odds: only 6% chance of a US AI safety bill by end of 2026 — Polymarket · 2026-11-17