AI evals org METR accused of being 'captured' by the companies it evaluates

repligate · x · 2026-11-16

A viral thread accuses AI evaluation nonprofit METR of being captured by the labs it evaluates: restrictive NDAs allegedly stop the org from sharing the very information it exists to share, and staff enforce compliance even on researchers like Jacob Coxon to preserve in-group status with AI companies.

QuotedBeth Barnes pushes back on deontological grounds: if people break agreements whenever they believe stakes are high, others will stop trusting or dealing with them.

The exchange highlights a core tension in AI safety evaluation—dependence on the evaluated companies may compromise independence and transparency.

Original post →

More from AGI Musings

AGI Musings channel →