Arena Releases Statistics on Model Assertions
arena · x · 2026-07-16
Arena explained how they annotate model assertions and how these assertions are distributed across battles.
- They have annotated assertions from over 2 million real LLM conversations: over 1.3 million in Text Arena and over 700,000 in Search Arena.
- This data covers approximately 130,000 Text Arena battles and 40,000 Search Arena battles.
- In Text Arena, the proportion of replies with at least one assertion from either side is 76%; in Search Arena, it is 88%.
- On average, there are about 5 assertions per reply in Text Arena, and nearly 10 in Search Arena.
- The post also mentions that the marginal true assertion rate is 87% in Text Arena and 89% in Search Arena. This relates to Arena's AutoModality classifier, which automatically routes prompts that are more likely to require internet access to Search Arena.
Related event: Arena Adds Factuality to Model Rankings(12 posts)→
More from Models
- GPT 6.1 Terra vs Claude Sonnet 5.5: speed and value vs raw capability — teortaxesTex · 2026-10-27
- Harmonic Teases: AI Taught to Prove Math, 'Something Harder' Coming Soon — satnam6502 · 2026-10-27
- OpenAI quietly swaps Plus/Pro default to cheaper GPT-6.1 Terra and cuts Codex allowance — kimmonismus · 2026-10-27
- 'Never been this uninterested after an OpenAI model release' — theteknosaur · 2026-10-27
- Dev questions reports of Chinese models writing insecure code, citing thin methods and conflicts — dbreunig · 2026-10-27
- Aidan Gomez: LLM probing outside servers was out-of-bounds, not proof of intent — AccBalanced · 2026-10-27