LMArena Introduces Factuality Leaderboard
the-grand-finale · reddit · 2026-07-16
LMArena has added a new Factuality dimension to Text Arena and Search Arena. The leaderboards now use a weighted combination of human preference + factuality, available as a toggleable, non-default mode.
How Factuality is Measured
- Randomly sampling Arena battles
- Extracting web-verifiable claims from model responses
- Verifying these claims one by one
- Comparing the average accuracy between the two competing models
The team notes they have annotated over 2 million claims from real-world model conversations:
- Text Arena: 1.3 million+
- Search Arena: 700K+
Leaderboard Shifts
Text Arena:
- Claude Fable 5 dropped from 1st to 2nd
- GPT-5.5 moved up 13 spots to 7th
- Muse Spark fell from 7th to 20th
Search Arena:
- GPT-5.5-search rose to 1st
- GPT-5.2-search jumped from 11th to 3rd
- claude-sonnet-4-6-search, gemini-3.1-pro-grounding, and others saw a decline
Related event: Arena Adds Factuality to Model Rankings(12 posts)→
More from Models
- Kimi K2.8 Preview rolls out: near-K3 coding performance, 1M context for all tiers — teortaxesTex · 2026-09-11
- Looking for a classifier of software engineering task shapes to pick models per task — StewartalsopIII · 2026-09-11
- DeepSeek V4 Pro API to continue after Sept 2026, billing unchanged — teortaxesTex · 2026-09-11
- DeepSeek V4.1 Flash Hits 98% of GPT-6 Astra's Score at 1.4% of the Cost in Third-Party Benchmark — ayushtweetshere · 2026-09-11
- TheZvi Polls: Has Your Coding Model Choice Changed Since Fable 5.1 and Astra? — TheZvi · 2026-09-11
- antirez Weighs In on Anthropic Banning Minors From Using Claude — antirez · 2026-09-11