Pointing-at-things eval: Gemini 4 Argon leads, Mistral Large 4's SOTA vision claim falls flat
pbaylies · x · 2026-10-15
A developer benchmark on everyday-photo pointing with scene understanding (not plain object description) yields surprises: Gemini 4 Argon scores 71.8 (10s) but isn't dominant; Mistral Large 4, marketed as SOTA vision, manages only 50.8 at 161s; Claude Haiku 5.5 (62.9) vastly outperforms Sonnet 5.5 (37.2), which suffers from false positives. Moondream 3.1 posts 59.4 at ultra-low cost; GPT-5.6 Terra bottoms out at 5.2.
More from Models
- Your prompt may carry everything you've ever publicly written online, argues kromem — repligate · 2026-10-15
- TypeSafe's Jev 1.14.0 unlocks multimodality, a new box type, and 262K context behind danger-zone flags — op7418 · 2026-10-15
- Gemini 4 Argon impresses on capability but users slam its overprotective safety guardrails — RexDouglass · 2026-10-15
- Child-safety benchmark KORA spurs real gains: Mistral jumps from last to 10th of 42 — GregKamradt · 2026-10-15
- Gemini 4 Argon coding test: token-hungry and pricey, $20 plans no longer buy frontier models — bytebot · 2026-10-15
- VC claims GPT-6 Astra and Opus 5.5 will do for creative work what Opus 4.5 did for code — venturetwins · 2026-10-15