Step 5 Preview scores 44 on Artificial Analysis: unmatched cost, weak on agents

On September 22, Artificial Analysis published its full evaluation of StepFun's new flagship Step 5 Preview: the model scored 44 overall, on par with Kimi K3 (max), slightly behind GLM-5.3 (max, 45) and Qwen3.8 Max, ranking 26th among 202 models on the leaderboard. Overall, it's a flagship whose biggest selling point is extremely low pricing, with leading reasoning ability but comparatively weak agent capabilities.

Confirmed

• Specs and overall score: 600B total parameters / 27B activated, Intelligence Index of 44, ranked 26th; output speed of 92.8 tokens/sec, faster than the 71 average; priced at $1/$2.70 per million tokens, both below the median.
• Reasoning and agent performance: leading reasoning, but across-the-board weakness on agent tasks is a clear shortcoming—GDPval-AA 1,566 Elo, AA-Briefcase 1,4xx (the post didn't give the full number).
• Knowledge and hallucination: AA-Omniscience accuracy of 42%, beating GLM-5.3 (max, 753B, 34%) and slightly above the factual-recall trend line for its parameter count; however, 43% of its answers contained hallucinations.
• Cost advantage: running the full Intelligence Index costs about $0.72 per task, versus roughly $2.00 for the same-scoring Kimi K3 (max) and even more for the 1-point-higher GLM-5.3 (max); Artificial Analysis notes it breaks the cost line through pricing rather than token efficiency, at roughly 1/2.8 the cost of peer models.

Why it matters

Step 5 Preview charts a differentiated path: rather than winning on absolute scores, it targets the value-for-money segment with 44-point intelligence at roughly 1/2.8 the per-task cost. But its 43% hallucination rate and agent weaknesses suggest caution is warranted for automation scenarios requiring high reliability.

2026-09-22 ~ 2026-09-22 · 6 related posts

Full story(2 episodes)→

• Episode 1: StepFun launches Step 5 Preview, touting cost-performance frontier(2026-09-21, 3 posts)
• Episode 2: Step 5 Preview scores 44 on Artificial Analysis: unmatched cost, weak on agents(2026-09-22, 6 posts)

• [source] StepFun's Step 5 Preview scores 44 on AA Intelligence Index at ~2.8x lower cost than Kimi K3 — ArtificialAnlys · 2026-09-22
• StepFun's Step 5 Preview matches Kimi K3 at ~2.8x lower cost per task — ArtificialAnlys · 2026-09-22
• [source] Step 5 Preview costs $0.72 per AA task vs $2.00 for Kimi K3 — pure pricing, not token usage — ArtificialAnlys · 2026-09-22
• Step 5 Preview beats GLM-5.3 on factual recall but hallucinates on 43% of attempts — ArtificialAnlys · 2026-09-22
• Step 5 Preview detailed breakdown: strong reasoning, weak agentic scores — ArtificialAnlys · 2026-09-22
• [source] Step 5 Preview lands at #26 of 202 on AA leaderboard with competitive pricing — ArtificialAnlys · 2026-09-22