Miles Brundage's AVERI publishes key definitions for frontier AI auditing and evaluation
Miles_Brundage · x · 2026-11-17
Miles Brundage says AI auditing/evaluation/assessment terminology is chaotic, and points to AVERI's new definitions page.
- Frontier: a general-purpose system performing at least as well across broad capability benchmarks as the best systems did a year earlier, including surrounding tools/software; a frontier developer released one in the past year or is building one.
- Assessment: umbrella term for any analysis of AI systems/companies, via evaluation, verification, or both, possibly using non-public info.
- Evaluation: generating new safety/security evidence about a system or developer.
- Framed as practical-use definitions (not for legislation), with an interactive chart of what currently counts as "frontier," to be updated over time.
More from AGI Musings
- Profound CTO dbabbs: software engineering isn't cooked — CS will matter most in the AI era — soleio · 2026-11-17
- Why the legal system's alignment toolkit fails for AI: copies beat prisons, penalties don't deter — trevposts · 2026-11-17
- As AI aced homework, calls grow for universities to build AI-free exam infrastructure — _akpiper · 2026-11-17
- AI raised the ambition floor: researcher now lets optimization runs continue past the first win — generativist · 2026-11-17
- Why the AI safety movement is hard to beat: it lures Jane Street traders to $60k Hill jobs — panickssery · 2026-11-17
- CAIS and Scale AI Launch Remote Labor Index to Measure AI Automation of Remote Work — ben_j_todd · 2026-11-17