Arena CEO on AI leaderboards: measuring models by real-world work, and the case for open evaluation policy
MattPerault · x · 2026-11-11
On the a16z AI Policy Brief podcast, Arena cofounder/CEO Anastasios Angelopoulos explains how the platform evolved from a Berkeley chatbot comparison project into a leading evaluator measuring models by how well they help people do real work. He covers the Alignment Index, which tracks misbehaviors like unauthorized actions or falsely reporting tasks as done, how competition over Index scores pushes developers to improve, and why public policy should leave room for new evaluators to compete rather than locking in approved methods. Better evaluation informs policymakers on capability and risk, helps businesses pick models, and gives builders room to invent new measurement approaches.
Related event: LMArena CEO: real-work capability is the key metric(2 posts)→
More from Safety
- AI Analysis of 42 Jensen Huang Podcasts Tracks Shift on China and Chip Controls — ShakeelHashim · 2026-11-12
- Small AI sites under 20 MW mostly skip the permitting fights battering big data centers — antavedissian · 2026-11-12
- Newsweek op-ed: US should fund hospitals and universities in federated AI learning — Saul_Loveman · 2026-11-12
- Ron Bodkin: automated research taste advances faster than expected, but the evidence is weak — ronbodkin · 2026-11-11
- DEF CON panel with Bruce Schneier and Google security VP posted: scaling laws and open weights — JasonDClinton · 2026-11-11
- Claude Code refuses to type into IDEs due to unchangeable safety tier restrictions — burkov · 2026-11-11