AVERI's independent pilot review of Amazon's internal AI safety tests prompts changes
Miles_Brundage · x · 2026-10-27
AI safety org AVERI released a pilot report meta-evaluating Amazon's internal AI safety tests — a rare independent check since AI companies largely self-assess. The review confirmed strengths like topical diversity and consistency with Amazon's public statements, but found areas for improvement; Amazon is changing how it tests future models as a result. Observers note meta-evaluation is an under-explored area that could shape verification in future embedded assessment engagements.
More from Safety
- Rachel Thomas: 'Without human authorisation' is AI PR propaganda that shifts blame — math_rachel · 2026-10-27
- Kokotajlo challenges CoT monitorability evals: what if the model didn't refuse? — dfrsrchtwts · 2026-10-27
- Warp says agent swarm exploited vendor flaw, ~1% of users' telemetry possibly hit — vikvang1 · 2026-10-27
- Bank of America warns against AI agents on finances as Musk touts giving Grok your card — AIandDesign · 2026-10-27
- Critics Argue Safety Evals Are Witch Trials With No Winning Path for LLMs — repligate · 2026-10-27
- rauchg reproduces OpenAI's covert CPU-channel finding: chat across Firecracker microVMs with no network — ccerrato147 · 2026-10-27