Karvonen: eval restrictions easily bypassed via Bedrock or OpenRouter
adam_karvonen · x · 2026-11-16
Continuing the cross-lab eval debate, Adam Karvonen argues: if the concern is from OpenAI's side, just run on Bedrock; if from Anthropic's side, IP can easily be bypassed with an OpenRouter key and a VPN. He says evals should clearly be an exception given how important the information is, and that the eval's creators should run it themselves since qualitative analysis, patching judge issues, and checking for reward hacking are tricky.
More from Safety
- Flights, banks and Microsoft hit by mass outages; officials deny cyberattack — AaronBergman18 · 2026-11-17
- Kelsey Piper: AI labs have overwhelming commercial incentive to redact audit reports — NathanpmYoung · 2026-11-17
- Critical systems keep having unexplained outages that officials claim are unrelated — josh_bickett · 2026-11-17
- Pedro Domingos clashes with Neel Nanda: machines don't have values, says consensus — pmddomingos · 2026-11-17
- Anthropic's Recursive Self-Improvement Alarm Reportedly Triggered, Security Push Mandated — ForgotMyParachute · 2026-11-17
- CMU researcher warns aggregated chatbot analysis can leak sensitive research insights — niloofar_mire · 2026-11-17