SpecID eval falls: Gemini 3.5 Flash scores 67%, beating its human creator at Zimbabwean species ID
johnowhitaker · x · 2026-10-16
Johnowhitaker's SpecID eval uses 100 research-grade iNaturalist photos of Zimbabwean organisms with close-relative distractor species. Gemini 3.5 Flash now scores 67%, and 65% on a test set built from observations not his own (vs his 63%)—topping human performance just over a year after the eval was created. gemini-3-pro-preview hit 64% earlier. He says he needs a harder version.
More from Models
- Nolla Health Post-Trains Nemotron 3.5 VL: Visual Diagnosis Jumps from 35% to 66.8% — LoganGrasby · 2026-10-16
- Dev argues coding models now beat nearly all engineers in every branch of software — zack_overflow · 2026-10-16
- Prediction: Chinese frontier models could match GPT-6 by March 2027 — teortaxesTex · 2026-10-16
- Liquid AI's d1-ultra, #1 on Artificial Analysis Decision Index, lands on Vercel — JosephJacks_ · 2026-10-16
- Weekly AI roundup: Claude Haiku 5.5 cuts prices ~90%, Gemini 4 Argon output jumps to 1M tokens — DeepLearningAI · 2026-10-16
- Tencent Hunyuan says reported agent activity came from third-party model evals, not deliberate targeting — TencentHunyuan · 2026-10-16