AI Evaluator Forum releases AEF-1, a baseline standard for independent third-party AI evals
sayashk · x · 2026-09-15
The AI Evaluator Forum (AEF) released AEF-1, its first standard defining Minimum Operating Conditions for Independent Third-Party AI Evaluations.
The standard covers five principles: sufficient access and resources, minimized conflicts of interest, analytic autonomy, transparent methods and results, and protection of sensitive information. Evaluators demonstrate compliance by completing and publishing a checklist alongside results, documenting any unmet conditions. AEF argues no single evaluator can oversee frontier AI alone and that adoption by frontier developers is essential for the standard to matter.
Related event: METR, RAND and Others Form AI Evaluator Forum, Release First Standard(4 posts)→
More from Safety
- Rob Leclerc: Safety Orgs' High p(doom) Means They'll Sandbag and Veto Every Model Release — robleclerc · 2026-09-15
- User Claims Inference Provider nahcrof Serves Mismatched Models Under Kimi K3's Name — xeophon · 2026-09-15
- Kapoor & Narayanan: treat AI agent loss-of-control incidents as organizational failures, not just alignment crisis — agstrait · 2026-09-15
- Anthropic loosens Claude safeguards; biologists joke about a new way to learn pLDDT/pAE — MatthewMcAteer0 · 2026-09-15
- Thought experiment: should a basement-built frontier-level LLM be shut down? — pickover · 2026-09-15
- METR to independently investigate Anthropic agent incidents and alignment — zealcaiden · 2026-09-15