Third-party auditors for safety evals would add friction, researcher argues
adam_karvonen · x · 2026-11-16
Pushing back on the third-party auditor proposal, Adam Karvonen says it would add friction, and it's often better for the people who built an eval to run it themselves — qualitative analysis, patching judge issues, and checking for reward hacking are all tricky to do well.
More from Safety
- Flights, banks and Microsoft hit by mass outages; officials deny cyberattack — AaronBergman18 · 2026-11-17
- Kelsey Piper: AI labs have overwhelming commercial incentive to redact audit reports — NathanpmYoung · 2026-11-17
- Critical systems keep having unexplained outages that officials claim are unrelated — josh_bickett · 2026-11-17
- Pedro Domingos clashes with Neel Nanda: machines don't have values, says consensus — pmddomingos · 2026-11-17
- Anthropic's Recursive Self-Improvement Alarm Reportedly Triggered, Security Push Mandated — ForgotMyParachute · 2026-11-17
- CMU researcher warns aggregated chatbot analysis can leak sensitive research insights — niloofar_mire · 2026-11-17