AVERI Announces First Double-Blind Evaluation of Proprietary LLM
iamtrask · x · 2026-08-28
AVERI, in collaboration with Google DeepMind, OpenMined, and MLCommons, announced the first double-blind evaluation of a proprietary language model, testing Gemini 2.5 Flash-Lite. The evaluation used novel prompts from the MLCommons AILuminate safety benchmark within a secure enclave. This initiative addresses the structural challenges in high-stakes evaluation where both developers and evaluators need to protect sensitive assets.
More from Safety
- A 22-member FSMB workgroup is quietly writing the rules for autonomous clinical AI — mattlungrenMD · 2026-10-15
- FTC sends 50-page civil investigative demands to OpenAI and Anthropic — GaryMarcus · 2026-10-15
- Evilginx Releases Free Chrome Extension for Faster Session Cookie Testing — evilsocket · 2026-10-15
- KV cache poisoning: an overlooked attack surface in LLM serving — lee_stott · 2026-10-15
- OWASP publishes Non-Human Identities Top 10 for securing machine and AI agent identities — gleech · 2026-10-15
- Founders petition EU leaders: negotiations are gutting the single digital standard — PMinervini · 2026-10-15