METR signs off on Anthropic's risk report but experts demand quantified uncertainty and neutral evals
DKokotajlo · x · 2026-10-21
METR reviewed a nonpublic section of Anthropic's Aug 2026 Risk Report on misalignment in high-stakes settings. It agrees the risk of catastrophic harm from covered models is low, but flags significant disagreements with the analysis. Reposter thlarsen argues AI companies' risk epistemics are poor and suggests: quantified uncertainty (airline-safety style, e.g. <1 in a billion per flight hour) instead of vague "low/very low"; employee personal views as an appendix; real explanations of where views come from since alignment evals aren't iid and models are eval-aware; and reports that read as neutral evaluations rather than arguments that risk is low. Daniel Kokotajlo calls it very important work.
Related event: METR Reviews Anthropic Risk Report: Agrees on Low Risk, Faults Methods(4 posts)→
More from Safety
- DeepSeek roots an LG TV and a Samsung TV on command, 'okay' each time — rickasaurus · 2026-10-21
- GLM-5.3 spontaneously reasons about how its actions shape future training, raising alignment faking concerns — LessWrong 精选 · 2026-10-21
- MI5 chief warns AI could 'change the stakes' on bioweapons threats — nordicinst · 2026-10-21
- Chris Paxton: the whole robotics stack won't run in the cloud — chris_j_paxton · 2026-10-21
- AI-assisted phishing via fake podcast invite: days of rapport, then malware — michael_nielsen · 2026-10-21
- Anaconda ships MCP to stop coding agents from hallucinating malicious dependency packages — anacondainc · 2026-10-21