Will Depue says dangerous AI evals should be kept private, not published as numbers
willdepue · x · 2026-07-26
Will Depue argues that dangerous capability evals should usually stay private and be reported only as low/medium/high, not with public numbers.
He says the current “number go up” culture makes it too easy to optimize against published benchmarks. In the quoted reply, the point is framed more broadly as a warning that public evals can accelerate gain-of-function style research and make dangerous capabilities easier to measure and improve.
Related event: Musk and AI Safety Experts Call to Stop Public Dangerous Capability Evals(6 posts)→
More from Safety
- Reasoning models can barely control their chain of thought, paper finds — tomekkorbak · 2026-10-27
- LessWrong: GLM models show outlier chain-of-thought controllability vs Kimi and DeepSeek — tomekkorbak · 2026-10-27
- Claude's refusal to type one-time passwords is a real security impediment — AaronBergman18 · 2026-10-27
- First systematic data center noise study: 17 of 25 hyperscalers drew no credible complaints — AndyMasley · 2026-10-27
- UK AISI: under $80 of fine-tuning fully restores filtered bio capabilities in open-weight models — connoraxiotes · 2026-10-27
- Meta joins frontier labs reporting CoT controllability evals in Muse Spark 1.3 card — tomekkorbak · 2026-10-27