Google's new Gemini Decision 1 ranks 19/54 on third-party JevBench at $0.038 per 1,000 decisions
airesearch12 · x · 2026-11-13
Benchmark Heaven (JevBench v1.8.10) has added Google's new Gemini Decision 1 (hosted API), with a vendor-vs-independent gap analysis:
- Scores: 19th of 54 hosted APIs on Composite Score; 13th of 39 on Capability.
- Self-reported vs measured: Google reports 92.2% on 231 public v1.2 items; the benchmark's 300-item run gets 87.3% plain accuracy (83.7% under a stricter yes/no rule), and 86.3% across all 1,500 items (1,200 non-public). Calibration is also strong on the shared board scale.
- Cost: $0.038 per 1,000 decisions at list price — 31 of the other 53 hosted APIs are cheaper; in the capability-vs-cost scatter it sits mid-pack behind options like Torchcast Decision 31B and decide-1.1 (85.3%).
More from Models
- Google AI Mode now enforces usage limits, users surprised by rate-limit popup — elvano124 · 2026-11-22
- Sol 6.2 now refuses reverse engineering that Sol 6.1 happily did for a month — cardak98 · 2026-11-22
- Anthropic's record ~$81B IPO, GPT-6.2 pricing, Qwen4-Max and Copilot Autopilot: week in review — ksraj1001 · 2026-11-22
- Reddit rant: Frontier AI labs are training models to imitate lazy engineers — RossPeili · 2026-11-22
- MBZUAI open-sources PAN 2, a 29B-parameter general-purpose world model — LinusEkenstam · 2026-11-22
- Inkling Mini turns out surprisingly fluent in Russian, sparking data recipe questions — teortaxesTex · 2026-11-22