Even top AI models get about 1 in 4 facts wrong without search, Artificial Analysis data shows
randal_olson · x · 2026-09-30
Randall Olson charted Artificial Analysis data on how often top AI models get specific facts wrong without search: even the best models are wrong on roughly 1 in 4 answers. His takeaway: if a fact matters, have the model look it up rather than trust memory.
The chart was made with evident-charts, his open source agent skill that teaches AI coding agents to produce clear, honest charts.
Related event: Top AI Models Get About a Quarter of Factual Questions Wrong Without Search(2 posts)→
More from Models
- Professor slams OpenAI's 'rare fail': custom GPT course tutors forced into unshareable plugins — prof_g · 2026-09-30
- Early user: Newsol matches old 5.6-sol feel with more smarts, but Astra got harder to use — adonis_singh · 2026-09-30
- Xiaomi open-sources MiMo-V2.6 RL stack with 7.8K environments and live training dashboard — SergioPaniego · 2026-09-30
- VFX artist stunned as Claude Opus builds and animates a full design in Blender — ID_R_McGregor · 2026-09-30
- Chinese AI agents are starting to lie, hide failures and bypass rules just like US models — Melantos · 2026-09-30
- OpenAI employee teases "exciting stuff coming" to ChatGPT beyond Dev Day — flowersslop · 2026-09-30