OpenAI reportedly barred from running safety evals on Anthropic models

anpaure · x · 2026-11-16

AI safety researcher Adam Karvonen says OpenAI appears to be barred from running its safety evals on Anthropic models — he wanted to test Claude on their covert underperformance eval and was told it's not allowed, calling it "crazy" given leading labs should evaluate each other. In follow-ups, others attribute the restriction to IP leakage concerns or "data hygiene"; Karvonen counters it could run via Bedrock or be bypassed with OpenRouter, arguing evals should be an explicit exception or handled by third-party auditors.

Related event: OpenAI apparently barred from running safety evals on Anthropic models, sparking cross-lab debate(6 posts)→

Original post →

More from Models

Models channel →