deepfates: good that METR gets more access, bad that evaluators are a monoculture
deepfates · x · 2026-09-13
Developer deepfates reacts to METR gaining more model access and third-party evaluators being consulted: the direction is good, but it's probably bad that evaluators come from a very tight monoculture — a capsule critique of the homogeneity of frontier-model safety evaluation.
More from Safety
- Tyler Alterman on the hacking panic: more Y2K than COVID, use 1Password and update software — jd_pressman · 2026-10-31
- Inference-only chip proposal would bar RL rollouts for models in training — wfithian · 2026-10-31
- Every Used Codex and Agents to Survive the Railway Credential Breach — every · 2026-10-31
- Researchers debate whether more inference compute can substitute for training compute — teortaxesTex · 2026-10-31
- OpenAI blocks weapons details for teens, adds auto-ending chats fixated on violence — rohanpaul_ai · 2026-10-31
- First machine-checked security proof for end-to-end x86 process execution published — banteg · 2026-10-31