October model wrap: 15 releases evaluated with 500B+ tokens, Kimi K3.1 Pro tops open-weight

BenBlaiszik · x · 2026-11-02

The Vals team spent 500B+ tokens evaluating the 15 new models released in October (15 models, 8 open-weight, 1 new #1):

Google DeepMind is now shipping near-SOTA models: Gemini 4 Carbon (70.4, #2 overall) and the not-yet-GA'd Argon;
Three Chinese releases: DeepSeek V4.1 Flash 1017 (61.3, best open-weight at launch, notably stronger in cybersecurity), MiniMax M3.1 Flash (53.8), and Kimi K3.1 Pro (62.5, now the #1 open-weight model on the Vals Index); GLM Legal tops open-weight LegalBench (86.2%);
Meta shipped Wonder, Wonder Full (66.8, #5), and Muse Spark Open (816B total / 42B active), among the most capable US open-weight models;
Reflection AI's Beam 1.0: 500B total / 23B active params with a 1M context window, strengthening the US open-weight ecosystem;
Others include Claude Haiku 5.5 (58.7), GPT-6.1 Terra (60.5), Mistral Large 4 OSS (56.4), NVIDIA Nemotron 3.5 Super VL, and Inception Mercury 3.
Vals Index moved to v2.2 on Oct 19; earlier scores were de-weighted from v2.1, so older debuts score lower.

More from Models