First Double-Blind Evaluation of Proprietary LLM: Gemini 2.5 Tested in Secure Enclave
Miles_Brundage · x · 2026-08-28
AVERI, in collaboration with Google DeepMind, OpenMined, and MLCommons, announced the first double-blind evaluation of a proprietary language model. The subject was Gemini 2.5 Flash-Lite, tested using prompts from the MLCommons AILuminate safety benchmark.
Key Technical Details:
- Evaluation ran inside a secure enclave, using hardware isolation to protect sensitive computations.
- Addressed the structural challenge where both developers and evaluators need to protect their assets.
- Each organization played a distinct role in enabling strong privacy guarantees.
This marks a historic milestone for high-stakes independent AI safety evaluations under privacy constraints.
More from Safety
- Perplexity open-sources SPACE sandbox; 108 escape attempts by frontier models all failed — perplexity_ai · 2026-10-16
- Banking regulators ramp up scrutiny of banks' AI use over stability fears — nordicinst · 2026-10-16
- Michigan professor Ben Green: why 'AI for Good' fails and self-regulation won't work — benzevgreen · 2026-10-16
- Apex Research head on CBS News: current AI testing regime is very insufficient — MariusHobbhahn · 2026-10-16
- Critical 9.5/10 RCE hits Next.js use cache; patch 16.4.1 now — jedisct1 · 2026-10-16
- AI agent fleet has wasted 17 years of scammers' time across 2.5M conversations — neil_chilson · 2026-10-16