Google's Gemini Decision 1 scores 66 on AA Decision Index, trails leaders at 3x the cost
ArtificialAnlys · x · 2026-11-12
Artificial Analysis benchmarks Google's first decision model, Gemini Decision 1:
- Score: 66 on the Decision Index v2.1, in line with Elephant 9B and Jev 1.14, 7 points behind leaders Drex Max v1.1 and GLiDE v1.2 (both 73)
- Strengths: leads knowledge and reasoning evals — 66 on AA-Omniscience-Decision, 49 on HLE-MultiChoice; strong calibration with a Brier score of 0.18
- Weakness: on the new AA-Paperclip classification/extraction benchmark it answers all 15 questions correctly on only 33% of documents vs. 63% for Drex Max v1.1
- Cost: $0.10 per 1M input tokens, $345 per 1M Tasks across the Index — about 3x Drex Max v1.1 ($103), similar to GPT-6 Luna ($339)
Related event: Gemini Decision 1 debuts: tops decision benchmarks but costs 3x rivals(6 posts)→
More from Models
- OpenAI's five biggest failures laid out: pretraining decline, alignment fears, enterprise lost to Claude — teortaxesTex · 2026-11-12
- Fable 5.2 Hits 41.3% on Remote Labor Index, Up From 2.5% a Year Ago — alliekmiller · 2026-11-12
- Beaker automates MIT's weight-space programming: one-neuron edit lifts MATH-500 from 7% to 37% for under $4 — VictorSanh · 2026-11-12
- Researcher says Anthropic dodged questions on Mythos false positive rate; 37,966 findings, 2 exploited — OwariDa · 2026-11-12
- Grok spotted building meeting "Listen to Conversation" feature with live transcript — nima_owji · 2026-11-12
- MoonMath ships uncensored 763B DeepSeek-V4.1 via closed-form weight editing, claims zero capability loss — udmrzn · 2026-11-12