OpenAI reportedly barred from running safety evals on Anthropic models
anpaure · x · 2026-11-16
AI safety researcher Adam Karvonen says OpenAI appears to be barred from running its safety evals on Anthropic models — he wanted to test Claude on their covert underperformance eval and was told it's not allowed, calling it "crazy" given leading labs should evaluate each other. In follow-ups, others attribute the restriction to IP leakage concerns or "data hygiene"; Karvonen counters it could run via Bedrock or be bypassed with OpenRouter, arguing evals should be an explicit exception or handled by third-party auditors.
More from Models
- Gemini Argon and Carbon rumored at 12T was a misquote — models may be even larger — oran_ge · 2026-11-17
- Pangram near-zero false positives on human text, but 'should you care?' debate rages — ipeirotis · 2026-11-17
- Anthropic's TASTE benchmark hides a shortcut; strongest models fall below chance — dhadfieldmenell · 2026-11-17
- Gamma's 100M-user data: OpenAI leads efficiency, Anthropic nuanced understanding — omooretweets · 2026-11-17
- Google ships Gemini 4 Carbon in AI Mode but not in the Gemini app for Pro users — gaganghotra_ · 2026-11-17
- Ripple-65B Preview Debuts as US Open-Weight SOTA on Agentic Coding at 65B — Azaliamirh · 2026-11-17