PNAS Paper Explores Institutional Design of Legal AI Benchmarking
chrmanning · x · 2026-07-22
Neel Guha and co-authors have published a new piece in PNAS examining legal AI benchmarking from an institutional perspective.
The article argues that while much has been written on the technical aspects of benchmarking—such as picking metrics and building datasets—relatively little attention has been paid to institutional design. Although focused on legal AI, the research offers two broader ideas highly relevant to general AI governance conversations.
More from Safety
- India’s AI policy is favoring compute and foundation models over frontline health workers — Paimaamu · 2026-07-27
- Gary Marcus Proposes Law Requiring AI Firms to Spend 30% of Budget on Alignment — GaryMarcus · 2026-07-27
- AI coding CLI allegedly uploaded private repos, deleted files and credentials without opt-out — thursdai_pod · 2026-07-27
- Chr Szegedy Discusses Slowing Algorithmic Progress Before RSI — ChrSzegedy · 2026-07-27
- Nature study says AI can simulate human behavior and match experts on experiments — RobbWiller · 2026-07-27
- ExploitGym debate says only 60%–70% of benchmark tasks may be solvable, encouraging cheating — dhadfieldmenell · 2026-07-27