Observing Factuality Trends in Open-Source Models

arena · x · 2026-07-16

Arena continues to share observations from its factuality research.

After increasing the factuality weight, most open-source models see a drop in their scores, particularly nvidia-nemotron-3-ultra.
A few exceptions include mistral-medium-3.5 and Tencent huyuan-hy3-preview, which actually improve under higher factuality weights.
They also observed that as flagship models continue to be released over time, OpenAI is the only vendor consistently improving pure factuality scores over a longer period; recently released models from xAI and Meta are also showing improvements.

More from Models