DeepMind pilots double-blind evaluations to fix benchmark contamination
Miles_Brundage · x · 2026-08-27
Google DeepMind is piloting a "double-blind evaluation" framework to address benchmark contamination. By using a secure enclave that hides both test prompts and model weights, the framework ensures external safety and performance evaluations remain private, robust, and trustworthy. The first successful test has been conducted on a frontier proprietary model.
More from Safety
- Anthropic red team's first-ever model "abliteration" cut GLM-5.3 refusals from 90% to 3% for ~$4,400 — cephaloform · 2026-10-16
- Anthropic grants METR employee-like access as independent safety evaluators go public — AnthropicAI · 2026-10-16
- APEX-Harm benchmark: script-based injection attacks easily compromise agents in shared workspaces — KAUST · 2026-10-16
- Residual-stream forecasts predict agent violations before they happen, USC study shows — USC · 2026-10-16
- AI safety drama: risk research called a 'company effort,' team lead responds — logangraham · 2026-10-16
- Brave's Leo adds end-to-end encrypted chat with GLM via attested TEE — gnukeith · 2026-10-16