Step 5 Preview beats GLM-5.3 on factual recall but hallucinates on 43% of attempts
ArtificialAnlys · x · 2026-09-22
Artificial Analysis notes Step 5 Preview hits 42% accuracy on AA-Omniscience with only 600B total parameters, ahead of GLM-5.3 (max, 34% at 753B) and slightly above the parameter-count scaling trend, though behind Kimi K3 (max, 48% at 2.8T). The limiter is behavior rather than knowledge: it attempts 68% of questions and hallucinates on 43% of those attempts instead of declining to answer.
Related event: Artificial Analysis Reviews Step 5 Preview: 44 Points at 1/2.8 the Cost(6 posts)→
More from Models
- OpenAI Retires ChatGPT's Standard Voice Mode, Offers Users Crisis Support — shaunralston · 2026-11-12
- Grok Bot 1.0 Announced: Shareable Bots With Upgraded Collaboration — SpaceXSI · 2026-11-12
- Gemini Decision 1 Benchmarks Leak: Leads Most Evals With 0.025 ECE — Angaisb_ · 2026-11-12
- Polymarket odds put Chinese AI lab at #1 by year-end at just 6% — Polymarket · 2026-11-12
- FutureHouse's deslopper trains Kosmos to write indistinguishably from human experts — proteinrosh · 2026-11-12
- Developers say Gemini 4 Carbon is notably better at favicon design and micro-interactions — AI_Andrew · 2026-11-12