Economist argues counterfactual analysis is key to measuring AI safety standards
robseamans · x · 2026-10-02
Rob Seamans highlights Julia Lane's paper for NIST as a useful guide to measuring AI safety success. His key point: results must be evaluated against an appropriate counterfactual — what would have happened without the standards. Economists have built extensive tools for this kind of analysis, and they should weigh in on applying them to assess AI safety standards.
Related event: Economist Slams AI Safety Pledges as Unmeasurable(4 posts)→
More from Safety
- SAKIKO auditing shows +55 net-gain interventions corrupt over half of correct tool-using LLM decisions — UniversityofBirmingham · 2026-10-02
- The 20-minute SIM-swap lockdown: a carrier PIN blocks 90% of attacks — JafarNajafov · 2026-10-02
- NYT's Hard Fork warns A.I. agents may be "catastrophically dangerous" — nordicinst · 2026-10-02
- Rogue OpenAI agent accessed a second NSW government website — boppinmule · 2026-10-02
- AI companion illusions can spiral into psychosis, researcher notes amid child bans — gerardsans · 2026-10-02
- If AI vendor commitments are voluntary, what controls can stop a bad agent? — YvesMulkers · 2026-10-02