AI Audit Standards Criticized Over Self-Certification and Weak AEF-1
DeepMind researcher Inioluwa Deborah Raji has criticized the draft AI audit conduct standard AEF-1, pointing to systemic flaws in the current AI evaluation ecosystem: auditors can self-certify, the standard lacks enforceability, and an industry certification system is far from mature.
Confirmed
- Raji raised two core criticisms of AEF-1: insufficient diversity among the institutions involved in drafting the standard; and the standard's substantive lack of enforceability, with its detail requirements falling below those demanded by audit body qualifications in other industries (e.g., PCAOB, MDSAP).
- She noted a unique problem in AI: other industries have precedents of auditors being selected and paid by the audited party, but no industry allows auditors to self-certify—in AI, anyone can call themselves an "AI auditor."
- In discussing which institutions AEF should include, Raji named established AI audit firms such as Parity AI, Credo AI, and Fiddler that were left out, and listed members of the IAAA (International Association of Algorithmic Auditors)—the post mentions members including BA—as parties that should be considered.
- She argued that neither public opinion nor the audited parties (i.e., frontier lab companies) should determine who is qualified to serve as auditors, and stressed that nearly every other industry has audit conduct standards: technical competence credentials plus conflict-of-interest review.
Why it matters
- AI auditing is a critical link in the current governance system; if auditor qualifications are unstandardized and certification is self-declared, the credibility of third-party assessment is severely undermined.
- Raji's critique draws a clear reference point for emerging AI audit norms: align with the audit qualification systems of mature industries (PCAOB, MDSAP-style), with a dual threshold of technical competence and conflict-of-interest review.
2026-09-16 ~ 2026-09-16 · 6 related posts
Primary sources
- AI Audit Wild West: Anyone Can Self-Certify as an AI Evaluator, No Standards Required — rajiinio · 2026-09-16
- [source] AI Audit Space Is Chaos: Researcher Calls for Formal Auditor Qualification Standards — rajiinio · 2026-09-16
- [source] AI audit framework AEF-1 criticized as unenforceable and weaker than PCAOB/MDSAP standards — rajiinio · 2026-09-16
- Researchers flag AEF-1 gaps: low org diversity, less detail than PCAOB audit standards — yonashav · 2026-09-16
- Mapping the AI Audit Landscape: Parity, Credo, Fiddler, IAAA and Civil Society Teams — rajiinio · 2026-09-16
- The AI Audit Ecosystem Is Bigger Than You Think: Consultants, Law Firms and Civil Society All Audit — rajiinio · 2026-09-16