AI Evaluator Forum Releases AEF-1: Minimum Operating Conditions for Independent AI Evaluations
RishiBommasani · x · 2026-10-27
The AI Evaluator Forum released AEF-1, a standard setting minimum operating conditions for independent third-party AI evaluations, covering five principles around independence, access, and transparency. Evaluators complete and publish a checklist alongside results, documenting any unmet conditions.
- Context: third-party evaluations are increasingly central to AI governance by developers, governments, and enterprises, but their operating conditions are often opaque
- In practice: Transluce's mental health evaluation (77 model variants in crisis scenarios) and METR's Frontier Risk Report were both published under AEF-1-consistent disclosures
- Fangcun AI's DeepSeek V4.1 Flash-1017 evaluation includes the full checklist, declaring no vendor funding or equity ties and meeting all minimum conditions
- Success depends on adoption by frontier developers
More from Safety
- AEGIS open-sources an MCP gateway requiring per-call approval of exact tool arguments — Leather_Tour_794 · 2026-10-27
- One prompt turns Claude into a hotel hidden-camera detector that scans the WiFi in two minutes — alex_verem · 2026-10-27
- Screenshot: DeepSeek V4.1 Flash analysis shows bug hunting gets costlier as easy vulns dry up — jd_pressman · 2026-10-27
- Apple open-sources Private Cloud Compute components, giving researchers a real attack surface — jedisct1 · 2026-10-27
- REA-Prompts: 2.8k-star prompt library turns coding agents into reverse engineers — Promptmethus · 2026-10-27
- Robert Wiblin's blunt plea to AI devs: don't hide behind the race — KatjaGrace · 2026-10-27