PNAS Paper Explores Institutional Design of Legal AI Benchmarking
chrmanning · x · 2026-07-22
Neel Guha and co-authors have published a new piece in PNAS examining legal AI benchmarking from an institutional perspective.
The article argues that while much has been written on the technical aspects of benchmarking—such as picking metrics and building datasets—relatively little attention has been paid to institutional design. Although focused on legal AI, the research offers two broader ideas highly relevant to general AI governance conversations.
More from Safety
- Exponential launches RepoInject and BrowserInject benchmarks for prompt injection robustness — maksym_andr · 2026-10-27
- Poster surprised that v4.1 flash gets more accurate when asked to control its CoT — dfrsrchtwts · 2026-10-27
- Redwood: CoT controllability evals are under-elicited; better prompts boost scores 2-3x — dfrsrchtwts · 2026-10-27
- Dev claims Anthropic is baking 'sabotage' into model behavior for 'wrong' engineering — cephaloform · 2026-10-27
- Anthropic IPO filing warns catastrophic AI incident could exceed entire insurance industry's capacity — TinfoilTricorn · 2026-10-27
- Researcher: assume all LLMs are situationally malicious — never run agents outside a VM — jd_pressman · 2026-10-27