Artificial Analysis Reviews Step 5 Preview: 44 Points at 1/2.8 the Cost

On Sep 22, Artificial Analysis published its full evaluation of StepFun's new flagship Step 5 Preview: an Intelligence Index of 44, tying Kimi K3 (max), slightly below GLM-5.3 (max, 45) and Qwen3.8 Max, ranking 26th among 202 models. It is a flagship whose biggest selling point is aggressive pricing, with leading reasoning but weak agent capabilities.

Confirmed

• Specs and score: 600B total / 27B active parameters; Intelligence Index 44, ranked 26th; output speed 92.8 tokens/s (above the 71 average); pricing $1/$2.70 per million tokens, both below the median.
• Reasoning vs agents: reasoning is strong, but agent tasks are a clear weakness — GDPval-AA 1,566 Elo, AA-Briefcase 1,4xx (full figure not given in the posts).
• Knowledge and hallucination: 42% on AA-Omniscience, beating GLM-5.3 (max, 753B, 34%) and slightly above the recall trend line for its parameter count; 43% of its answers exhibit hallucination.
• Cost advantage: about $0.72 per Intelligence Index task versus $2.00 for Kimi K3 (max) at the same score; Artificial Analysis notes it beats the cost line via pricing rather than token efficiency, at roughly 1/2.8 the cost of peers.

Why it matters

Step 5 Preview charts a differentiated path: competing on price-performance — 44 points at about 1/2.8 the per-task cost — rather than absolute scores. Its 43% hallucination rate and agent weaknesses call for caution in high-reliability automation scenarios.

2026-09-22 ~ 2026-09-22 · 6 related posts

Full story(2 episodes)→

• Episode 1: StepFun Surprises with Step 5 Preview, Touting Cost-Performance Frontier(2026-09-21, 4 posts)
• Episode 2: Artificial Analysis Reviews Step 5 Preview: 44 Points at 1/2.8 the Cost(2026-09-22, 6 posts)

• [source] StepFun's Step 5 Preview scores 44 on AA Intelligence Index at ~2.8x lower cost than Kimi K3 — ArtificialAnlys · 2026-09-22
• StepFun's Step 5 Preview matches Kimi K3 at ~2.8x lower cost per task — ArtificialAnlys · 2026-09-22
• [source] Step 5 Preview costs $0.72 per AA task vs $2.00 for Kimi K3 — pure pricing, not token usage — ArtificialAnlys · 2026-09-22
• Step 5 Preview beats GLM-5.3 on factual recall but hallucinates on 43% of attempts — ArtificialAnlys · 2026-09-22
• Step 5 Preview detailed breakdown: strong reasoning, weak agentic scores — ArtificialAnlys · 2026-09-22
• [source] Step 5 Preview lands at #26 of 202 on AA leaderboard with competitive pricing — ArtificialAnlys · 2026-09-22