Consistency test: Kimi K3.1 Pro passes 45% of VIALS tasks on all 3 tries vs 63% for GPT-6 Astra
ArtificialAnlys · x · 2026-11-07
Artificial Analysis highlights that solving a task once doesn't mean consistent performance. On VIALS, Kimi K3.1 Pro answers 119 of 161 tasks correctly at least once across three attempts vs 118 for GPT-6 Astra — nearly a tie. But requiring all three attempts to succeed: Kimi passes 73 (45%) while Astra passes 101 (63%). Consistency, not one-off success, is what separates frontier models.
More from Models
- Meta ships Muse Wonder specialized models: 2.4x less forgetting, 67% better unsticking — AIatMeta · 2026-11-07
- Generated worlds tested at runtime: local interactions 83/100, stateful workflows just 30/100 — nptacek · 2026-11-07
- Steering a single neuron gives Qwen3-30B a new identity in ~1-2% of replies — Sauers_ · 2026-11-07
- OpenAI and Anthropic stuck in a release standoff as both hold back their next models — kimmonismus · 2026-11-07
- Deep Learning Weekly #480 ships with 'Gemini 4 Carbon' mention and model-routing paper — dl_weekly · 2026-11-07
- PostTrainBench: AI agents' post-training score jumps from 23% to 45% in 7 months — karinanguyen · 2026-11-07