Karvonen: eval restrictions easily bypassed via Bedrock or OpenRouter

adam_karvonen · x · 2026-11-16

Continuing the cross-lab eval debate, Adam Karvonen argues: if the concern is from OpenAI's side, just run on Bedrock; if from Anthropic's side, IP can easily be bypassed with an OpenRouter key and a VPN. He says evals should clearly be an exception given how important the information is, and that the eval's creators should run it themselves since qualitative analysis, patching judge issues, and checking for reward hacking are tricky.

Related event: OpenAI apparently barred from running safety evals on Anthropic models, sparking cross-lab debate(6 posts)→

Original post →

More from Safety

Safety channel →