AI safety researcher pushes back: academia does make sense as a model evaluator
dhadfieldmenell · x · 2026-09-13
Responding to Nathan Lambert's claim that academia is a poor fit as an independent AI evaluator, Dylan Hadfield-Menell argues that running evaluations on private models and data offers clear benefits to researchers, even with limits on sharing results.
He notes that evaluating and auditing frontier systems aligns directly with academics' stated goals, and that many scholars already spend 20% of their time in industry, channeling those insights into public-facing lab research. The core tension is incentives: Lambert sees academics' drive to publish as incompatible with handling private information, while Hadfield-Menell sees access to private model data as worth that trade-off, with pros and cons across a range of independent evaluators.
More from Safety
- Tyler Alterman on the hacking panic: more Y2K than COVID, use 1Password and update software — jd_pressman · 2026-10-31
- Security researcher: vibe coding will force software to consolidate around mature products — jd_pressman · 2026-10-31
- Inference-only chip proposal would bar RL rollouts for models in training — wfithian · 2026-10-31
- Every Used Codex and Agents to Survive the Railway Credential Breach — every · 2026-10-31
- Researchers debate whether more inference compute can substitute for training compute — teortaxesTex · 2026-10-31
- OpenAI blocks weapons details for teens, adds auto-ending chats fixated on violence — rohanpaul_ai · 2026-10-31