Solar Pro 4's Lower Hallucination Rate Comes from Abstention, Not Knowledge
ArtificialAnlys · x · 2026-08-13
Evaluation data shows that Solar Pro 4 improved its AA-Omniscience score from -53 to -1, with a significant drop in hallucination rate from 88% to 24%.
However, this improvement is largely driven by abstention rather than actual knowledge acquisition. Solar Pro 4 attempted to answer only 41% of questions (down from 92% for Solar Pro 3), while its AA-Omniscience Accuracy remained nearly unchanged at 19%.
More from Models
- Users Complain About Claude's Unstoppable Chain-of-Thought Output — GuyHachmon · 2026-08-13
- OpenAI's gpt-live-1 Achieves Near-Perfect Conversational Turn Detection — pbbakkum · 2026-08-13
- xAI Launches Grok 4.6 Across Cursor, API with 2x Token Promo — aman_madaan · 2026-08-13
- xAI Exec Hints at Grok 4.5: Focused on Coding Agents — aman_madaan · 2026-08-13
- Reddit Speculates: Is Grok 4.6 a Fine-tune of Kimi K3? — robertpro01 · 2026-08-13
- Grok 4.6 Tops Knowledge-Work, Productivity, and Legal Benchmarks — elonmusk · 2026-08-13