StepFun's Step 5 Preview scores 44 on AA Intelligence Index at ~2.8x lower cost than Kimi K3
ArtificialAnlys · x · 2026-09-22
Artificial Analysis published a full evaluation of StepFun's new flagship Step 5 Preview, scoring 44 on the Intelligence Index — matching Kimi K3 (max) and just behind GLM-5.3 (max, 45) and Qwen3.8 Max (45).
Highlights
- Cost edge: $0.72 per Intelligence Index task, 2.8x cheaper than Kimi K3 (max) at the same score ($2.00), driven by $1/$2.70 per 1M input/output token pricing. Only MiMo-V2.6-Pro scores higher (46) at a lower cost ($0.13).
- Big reasoning jump: 46% on Humanity's Last Exam (+25 pts vs Step 3.7 Flash) and 21% on CritPt (+19), roughly in line with Kimi K3.
- Knowledge vs. hallucination: 42% accuracy on AA-Omniscience at 600B total params, beating GLM-5.3 (753B), but it hallucinates on 43% of attempts instead of declining, capping its index at 16.
- Agentic weakness: lags peers on GDPval-AA (1,566 Elo), Terminal-Bench 4.0 (33%), and AA-Briefcase (1,432 Elo).
Specs: 600B total / 27B active MoE, 1M context, text/image/video input; closed-weights API now, open weights planned for October 15.
Related event: Artificial Analysis Reviews Step 5 Preview: 44 Points at 1/2.8 the Cost(6 posts)→
More from Infra
- Gas-for-AI-datacenters forum co-located with CoRL shows energy demand spillover — carlosdponx · 2026-11-12
- a16z podcast: Mike Volpi on Cisco's 70+ acquisition playbook and the AI buildout — a16z · 2026-11-12
- AMD to pass Intel in server revenue by mid-2027 as share gains 6 points a year — BenBajarin · 2026-11-12
- Study: Reflective Data Center Roofs Actually Cool Daytime Ground by 1.8-2.2°C — SumitGup · 2026-11-12
- Tsinghua's BaGuaLu-2 tames MoE expert-load spikes at runtime, accepted to SC26 — jiqizhixin · 2026-11-12
- antirez backs low-bit DeepSeek 4.1 Flash over Qwen 27B dense on Apple M5 Macs — antirez · 2026-11-12