METR Independence Under Fire: AI Audit Ecosystem Accused of Revolving Doors and Regulatory Capture

From September 12–14, a multi-party debate erupted over the independence of AI safety evaluators, especially METR, triggered by Anthropic's proposal to have METR voluntarily supervise its AI systems and by METR and Redwood gaining access to OpenAI-Hugging Face incident logs. The core takeaway: the audit ecosystem has structural independence problems in personnel ties, funding sources, ideological homogeneity and selection transparency; defenders call this an unavoidable structural reality, while sharper critics such as Heidy Khlaaf and Halvar Flake allege outright regulatory capture.

Confirmed

• Deborah Raji (rajiinio) noted that a METR investigator on the HF/OpenAI incident probe is married to an OpenAI board member, and that most access contracts include constraints on evaluations; she welcomed Anthropic's openness to external audit but stressed that a mature audit ecosystem requires auditor conduct standards.
• ruima (via kristoph) challenged Anthropic's Pacing the Frontier initiative for committing only to third-party oversight, not to slowing training or internal capability research.
• A user (via NielsRogge) highlighted the timeline: an Anthropic employee left to join METR, and the next day METR was nominated as Anthropic's third-party evaluator.
• Beff Jezos argued Redwood Research, METR and similar bodies are staffed by former Big Tech employees holding related stock options, predicting a revolving door that would yield regulation protecting insiders; he mocked lab safety evaluations as friends grading friends.
• evijit showed that nominally independent pre-release evaluators with early access are largely run by ex-lab employees and friends of lab founders (verifiable in recent system cards), and asked why only METR and Redwood got the OpenAI-HF logs rather than interested government bodies like CAISI and CISA.
• Melanie Mitchell raised the general concern that evaluators hoping to be hired or acquired by OpenAI cannot stay independent.
• An industry insider (via emax) said the independent evaluators come from the same EA circles, funded by the same Open Philanthropy money, with founders ex-OpenAI.
• @NathanB60857242 argued METR is formally independent but socially and culturally intertwined with Anthropic, with donor pools heavily tied to Anthropic equity—hence independence with an asterisk.
• Heidy Khlaaf (via sarahbmyers) accused Anthropic of regulatory capture, calling METR ideologically and financially bound to vendors and its CoT-based, selectively-evidenced methods unscientific.
• Halvar Flake (via basedjensen, AccBalanced) endorsed the critique, saying METR is neither independent nor scientific—its independence from frontier labs is legal-formal only, like Soviet republics holding UN seats.
• Gary Marcus said METR is great but not the answer; another commenter (via AlexTensor) called METR non-independent, one of Silicon Valley's most radical AI-safetyist/EA groups.
• Kyrannio attacked the childlike idealism around independent evaluators who will decide training, alignment and release, while almmaasoglu found the voluntary third-party supervision of everything hard to believe.

Defenses

• MaskedTorah told evijit this is a structural reality: safety researchers know each other, expertise inevitably overlaps with labs, and independent incident investigation needs no internal access.
• BronsonSchoen countered that METR's model access is extremely limited, suggesting labs aren't coddling evaluators, while noting labs could weaponize COI screening.
• Joshua Saxe, from four years in pre-release LLM safety testing, argued METR is a major step toward independence versus the status quo, though some criticism lacks collegial charity.

Why it matters

• External audits are a key mechanism for AI safety governance and frontier-slowdown policies, and this controversy exposed structural independence problems across personnel, funding, ideology and selection. As evijit stressed, when evaluations anchor international AI governance, what affects the public must be publicly checkable. With figures like Flake and Khlaaf escalating to regulatory-capture accusations and Gary Marcus joining the skepticism, the episode has pushed discussion toward auditor conduct standards and institutionalized independence safeguards.

2026-09-12 ~ 2026-09-14 · 31 related posts

Full story(20 episodes)→

• Episode 1: OpenAI employees speak out urging slower AI development over extinction risk(2026-09-10, 4 posts)
• Episode 2: OpenAI asks Congress whether coordinated AI slowdown would violate antitrust law(2026-09-11, 4 posts)
• Episode 3: Dario's "Pace the Frontier" Plea Draws Rare Backing from Altman and Musk(2026-09-11, 56 posts)
• Episode 4: Amodei, Altman and Musk unite on slowing AI, sparking debate over motives(2026-09-11, 62 posts)
• Episode 5: Anthropic CEO Dario Amodei Calls for Slowing Frontier AI Development(2026-09-12, 203 posts)
• Episode 6: METR Independence Under Fire: AI Audit Ecosystem Accused of Revolving Doors and Regulatory Capture(2026-09-12, 31 posts)
• Episode 7: Dario Amodei's "Pace the Frontier" Essay Draws Wave of Skepticism(2026-09-13, 30 posts)
• Episode 8: Amodei, Altman and Musk Rally Behind Slowing Frontier AI Development(2026-09-13, 20 posts)
• Episode 9: The Open Weights Regulation Debate: Tiered Controls, Ban Risks, and the Fate of Open-Source AI(2026-09-13, 18 posts)
• Episode 10: OpenAI researcher: taming superintelligence is science, not pacing deals(2026-09-13, 3 posts)
• Episode 11: 'China Will Slow Too If We Do' Claim Mocked as AI Race Meme Spreads(2026-09-13, 4 posts)
• Episode 12: Open-source camp pushes back on frontier slowdown pledges(2026-09-13, 4 posts)
• Episode 13: OpenAI researcher: labs' internal progress far exceeds outside perception(2026-09-13, 5 posts)
• Episode 14: Report: Anthropic and OpenAI May Be Slowing AI Research Ahead of IPOs(2026-09-13, 2 posts)
• Episode 15: François Chollet Questions Whether Frontier Labs' AI Slowdown Proposals Are Safety or Market Capture(2026-09-13, 6 posts)
• Episode 16: Coordinated AI Slowdowns Could Raise Antitrust Concerns(2026-09-13, 3 posts)
• Episode 17: Hassabis Backs Dario's AI Slowdown Call as Standards Bodies Weigh In(2026-09-13, 2 posts)
• Episode 18: Critics Call Out AI Labs' Safety Rhetoric as Hypocritical(2026-09-13, 3 posts)
• Episode 19: White House AI Chief Sacks Tells OpenAI and Anthropic to Slow Down Without Regulation(2026-09-13, 12 posts)
• Episode 20: Amodei Calls for Slowing Frontier AI and Opening Anthropic to Independent Safety Reviewers(2026-09-14, 9 posts)

• Deborah Raji flags audit conflict: METR investigator married to OpenAI board member — rajiinio · 2026-09-12
• METR as AI Evaluator Makes Sense, But Oversight Needs Ideological Diversity — Dr_Atoosa · 2026-09-13
• Third-party AI evals critiqued as insider clubs run by ex-lab employees — evijit · 2026-09-13
• Publicly consequential AI evals should be testable by the public, says researcher — evijit · 2026-09-13
• Critics question METR's independence in Anthropic's 'Pace the Frontier' pledge — kristoph · 2026-09-13
• Critics say independent AI evaluators share the same EA funding and OpenAI ties — emax · 2026-09-13
• Evaluator Independence May Become Labs' Talking Point for Regulator Shopping — BronsonSchoen · 2026-09-13
• Researchers clash over why only METR and Redwood got OAI-HF incident log access — evijit · 2026-09-13
• Safety Researcher Defends Independent AI Incident Probes Without Lab Access — MaskedTorah · 2026-09-13
• Researcher questions why only METR and Redwood got access to OpenAI-HF incident logs — evijit · 2026-09-13
• Melanie Mitchell asks how "independent" AI evaluators can escape the revolving door — MelMitchell1 · 2026-09-13
• Nathan Lambert on evaluator independence: overcommitted bloggers are a starting point — natolambert · 2026-09-13
• [source] RedWood/METR evals staffed with ex-lab insiders, warns of a "revolving door" regulatory cartel — beffjezos · 2026-09-13
• Anthropic employee joins METR; next day METR named third-party evaluator — NielsRogge · 2026-09-13
• Kyrannio slams 'independent evaluators': enough childlike idealism about AI audits — Kyrannio · 2026-09-13
• Kyrannio rails against independent AI evaluators: 'markets do not operate on good faith' — Kyrannio · 2026-09-13
• Anthropic's proposal to make METR a voluntary third-party overseer draws skepticism — almmaasoglu · 2026-09-13
• e/acc founder mocks frontier safety evals as friends grading friends — beffjezos · 2026-09-13
• Halvar Flake: METR's independence from frontier labs is only nominal — AccBalanced · 2026-09-13
• Halvar Flake slams METR as neither independent nor scientific, cites Anthropic regulatory capture — basedjensen · 2026-09-13
• ThePrimeagen claims 'METR is not independent'; ex-OpenAI safety lead amplifies — Miles_Brundage · 2026-09-14
• [source] Anthropic-backed METR auditor accused of regulatory capture and unscientific methods — sarahbmyers · 2026-09-14
• Gary Marcus Amplifies Attack on METR: "Great, But Not the Answer" for AI Oversight — AlexTensor · 2026-09-14
• AI Now paper warns of 'safety revisionism' weakening AI risk thresholds — sarahbmyers · 2026-09-14
• Debate: does METR's independence hinge on Anthropic share windfalls? — dylanbowmanSF · 2026-09-14
• Debate: METR should be counted as independent 'with an asterisk' — NathanB60857242 · 2026-09-14
• AI Safety Researchers Clash Over Lab Independence and Talent Tradeoffs — joshua_saxe · 2026-09-14
• halvarflake: Finding Safety Researchers Untainted by Labs Is Nearly Impossible — halvarflake · 2026-09-14
• Safety Researcher Defends METR's Independence in Model Testing Debate — joshua_saxe · 2026-09-14
• Ex-Safety Chief Lists Why Lab Self-Testing Falls Short of METR — joshua_saxe · 2026-09-14

1 near-duplicate retellings: rajiinio