Gemini Decision 1 tops knowledge and reasoning evals on calibration, at a steep cost
ArtificialAnlys · x · 2026-11-12
Artificial Analysis shared per-benchmark details for Gemini Decision 1:
- AA-Omniscience-Decision: Decision Score of 66, ahead of d1-ultra (64) and GLiDE v1.2 (61), level with GPT-6 Luna (Max); 45% raw accuracy vs. 28% for Drex Max v1.1 and 27% for GLiDE v1.2
- HLE-MultiChoice: leads at 49 (Mammoth 27B and d1-ultra both 47); accuracy matches Decider v1.2 and Jev 1.14, but its Brier score of 0.20 is among the lowest of all models, driving its Decision Score lead
- Cost: $345 per 1M Tasks vs. $103 for Drex Max v1.1, $68 for Decider v1.2, and $14 for Celeris-1.1 Decision
Related event: Gemini Decision 1 Tops Decision Benchmarks on Calibration(4 posts)→
More from Models
- Researcher says Anthropic dodged questions on Mythos false positive rate; 37,966 findings, 2 exploited — OwariDa · 2026-11-12
- Grok spotted building meeting "Listen to Conversation" feature with live transcript — nima_owji · 2026-11-12
- MoonMath ships uncensored 763B DeepSeek-V4.1 via closed-form weight editing, claims zero capability loss — udmrzn · 2026-11-12
- 11 LLMs battle in 1,168 Cambridge Union debates; Claude Fable 5.2 dominates at 209-14 — AymericRoucher · 2026-11-12
- Claude app widget bugs: edits invisible to Claude, mobile remote control broken — petergyang · 2026-11-12
- Allen-Zhu's new paper: Canon layers boost reasoning 2x, even random convolution weights help — teortaxesTex · 2026-11-12