DeepSeek 1017 Scores 24% on CritPt Physics Benchmark, Cost Savings Questioned

teortaxesTex · x · 2026-10-19

Artificial Analysis' CritPt physics reasoning leaderboard puts DeepSeek 1017 (max) at 24%, the top DeepSeek model and ahead of Kimi k3 and GLM-5 Preview, though well behind GPT-6, Claude Opus 5.5 and Gemini 3.1 Pro (31-33%).

Commenter teortaxesTex calls the results 'rather modest' and questions the claimed cost savings versus V4.1 Flash: the model is equally verbose on Artificial Analysis, and he suspects the benchmark may have mistakenly priced 1017 at off-peak rates.

Related event: DeepSeek V4.1-Flash-1017 Posts Lowest Hallucination Rate but Regresses on Knowledge and Long Context(3 posts)→

Original post →

More from Models

• Robotics Builder: GPT-6 with Controller-Level Function Calling Could Nail 100 Apple Pick-and-Placements by 2026 — wzenus · 2026-10-19
• Google Japan gives students Google AI Plus free for 12 months, AI Pro at ¥720 — heiga_zen · 2026-10-19
• Anthropic Commits 5% of Shares to Public Benefit Fund; OpenAI Aurora Models Spotted on Arena — WorldofAI · 2026-10-19
• Retrained "bee model" drifts into dreamy, detached trace-like outputs — Sauers_ · 2026-10-19
• One prompt in a normal ChatGPT chat yields an 87-second chalkboard explainer video — mallow610 · 2026-10-19
• Claude Opus normal chats burn code quota, user says ChatGPT sub is far more generous — mallow610 · 2026-10-19

Models channel →