Arena Leaderboard Adds Factuality Metrics

arena · x · 2026-07-16

Arena has updated its main text leaderboard to a weighted result combining human preference and factuality. Factuality is now available as a non-default toggle in both Text Arena and Search Arena.

Their approach involves randomly sampling battles, extracting web-verifiable claims, verifying them one by one, and comparing the average accuracy of model responses. To support this ranking, the team has annotated over 2 million claims made by LLMs in real conversations, including 1.3 million+ from Text Arena and 700,000+ from Search Arena.

Upon enabling factuality, the author observed the following:

OpenAI models are the most stable overall, with almost no drops due to factuality;
Anthropic and Google showed mixed results;
Meta's Muse Spark and Dola Seed dropped significantly, though Muse Spark 1.1 still shows clear progress over 1.0 and the Llama series.

Original post →

Arena Leaderboard Adds Factuality Metrics

More from Models