Artificial Analysis discloses full benchmarking methodology spanning a dozen evals
ArtificialAnlys · x · 2026-09-05
Artificial Analysis published the full methodology behind Intelligence Index v4.2, explaining how the index combines suites of reasoning, knowledge, math and coding datasets into an overall intelligence synthesis, while acknowledging the limits of any eval metric.
The page lists current evals: agentic (AA-Briefcase, GDPval-AA v2, 𝜏³-Banking), coding (Terminal-Bench v2.1, SciCode), general (AA-Omniscience, GDP.pdf, AA-LCR v1.1), scientific reasoning (HLE, CritPt) and more, plus scoring details like multiple-choice extraction regexes, equality-checker LLM prompts, and code extraction—alongside version history for legacy evals such as GPQA Diamond and MATH-500.
More from Research
- AI agents with $10,000 budgets still fail open-ended research: GPT-6 Astra and Fable 5.1 papers rejected — sayashk · 2026-10-21
- Stanford team launches ScholarCatalyst, a benchmark for literature retrieval that inspires research — lateinteraction · 2026-10-21
- Simple symmetric Venn diagrams with 17, 19 and 23 curves found via AI-assisted search, verified in Lean 4 — pickover · 2026-10-21
- Pathway CEO in Forbes: Transformers Won't Get Us to Superintelligence, Experience-Learning AI Will — schwentker · 2026-10-21
- Laude Institute Backs Stanford's CLMs: Contrastive Language Models for Fast Decision-Making — Azaliamirh · 2026-10-21
- OpenAI acquires Orchestra Bio to bolster life sciences AI model Rosalind — TechCrunch AI · 2026-10-21