ICML Paper Finds AI Social Impact Evaluations Shrinking

An ICML paper analyzing 186 first-party model cards and 248 third-party evaluations finds that disclosure of social impact assessments has shrunk significantly since ChatGPT's commercialization, with evaluators over-relying on benchmarks.

2026-09-14 ~ 2026-09-14 · 3 related posts

• Evals are vanishing from model cards since ChatGPT went mainstream — evijit · 2026-09-14
• ICML paper: AI firms' social impact disclosures are sparse and declining — evijit · 2026-09-14
• Eval orgs overindex on benchmarks, neglecting societal impacts — evijit · 2026-09-14