New Artificial Analysis V5 index shows science-heavy models sliding on knowledge evals
teortaxesTex · x · 2026-10-28
Reacting to Artificial Analysis's new V5 index, teortaxesTex notes that Gemini 3 (1017) fell off hard on science and knowledge benchmarks. He argues this is expected: those capabilities are scale-sensitive and Flash is an SWE-first model, so the team needs a stronger base model to recover the ground — and he believes they're working on it.
More from Models
- Gemini Notebook reportedly adding Connectors, with Google Drive as the only launch integration — testingcatalog · 2026-10-28
- IBM's STAIR retriever cuts hallucination 65x vs fine-tuned RAG, hits 82.6% Recall@1 — Promptmethus · 2026-10-28
- One prompt burned Gemini 4's entire 5-hour quota; Mammouth AI used just 4% — fazerdino · 2026-10-28
- Why OpenAI and Anthropic keep swapping places on reasoning evals — teortaxesTex · 2026-10-28
- Reddit debates the (unconfirmed) Gemini Argo Ultra usage limits — Qubit99 · 2026-10-28
- OpenBMB open-sources MiniCPM-OCR: 1B SOTA OCR plus a 2B accuracy-checker model — alifcoder · 2026-10-28