Google DeepMind pilots world's first double-blind evaluation for proprietary AI models
heiga_zen · x · 2026-08-28
To address the challenge of benchmark contamination in AI evaluations, Google DeepMind has launched the first double-blind evaluation pilot for proprietary models. Traditional methods often trade off between the risk of prompt leakage or exposing model IP weights. By leveraging Google Cloud's Confidential Space, DeepMind executes evaluations within a cryptographically secured environment. This allows third parties to verify models without accessing weights, while model providers cannot see the test questions. Collaborating with the Singapore AI Safety Institute, OpenMined, and others, the pilot tested Gemini Flash Lite on confidential benchmarks, aiming to significantly enhance evaluation integrity and trust in AI safety.
More from Safety
- A 22-member FSMB workgroup is quietly writing the rules for autonomous clinical AI — mattlungrenMD · 2026-10-15
- FTC sends 50-page civil investigative demands to OpenAI and Anthropic — GaryMarcus · 2026-10-15
- Evilginx Releases Free Chrome Extension for Faster Session Cookie Testing — evilsocket · 2026-10-15
- KV cache poisoning: an overlooked attack surface in LLM serving — lee_stott · 2026-10-15
- OWASP publishes Non-Human Identities Top 10 for securing machine and AI agent identities — gleech · 2026-10-15
- Founders petition EU leaders: negotiations are gutting the single digital standard — PMinervini · 2026-10-15