Artificial Analysis launches Decision Index v2.1 benchmarking models across six evaluations
ArtificialAnlys · x · 2026-11-14
Artificial Analysis released Decision Index v2.1, a composite benchmark measuring decision models across document decisions and extraction, tool calling, knowledge, hallucination, reasoning, and invariance.
The six sub-evaluations and weights:
- AA-Paperclip (30%): 1,500 resumes, receipts/invoices and research papers, each with 15 answers treated as one decision
- EvalSafe-ONET-AA (15%): workplace document judgments based on TypeSafe's ONET benchmark
- When2Call (10%): NVIDIA's tool-calling evaluation
- AA-Omniscience-Decision v1.1 (15%): private dataset adapted from the knowledge/hallucination benchmark
- HLE-MultiChoice v1.1 (15%): text-only multiple-choice questions from Humanity's Last Exam, with an option to decline
- AA-Symmetry v1.1 (15%): checks whether decisions hold when options are moved, reworded, or the question negated
The team says it will keep iterating the index to help developers and companies understand this emerging model class; full results are on its site.
More from Research
- NarrativeBench at NeurIPS: best LLMs cover only ~75% of available narratives — anas_ant · 2026-11-14
- TurboPairFormer open-source kernels speed up OpenFold3 training up to 1.73x — MoAlQuraishi · 2026-11-14
- Microsoft paper shows 7 edited neurons can backdoor an AI agent with a timed trigger — DimitrisPapail · 2026-11-14
- Sakana AI Keynote in Seoul: The Path to Recursive Self-Improving AI — Stefania_druga · 2026-11-14
- CoRL spotlight GaP debuts as open-source robotics sim RSS hits 500+ GitHub stars — erwincoumans · 2026-11-14
- CMU's EgoX: cross-embodiment egocentric manipulation via "embodiment dreaming" pretraining — CarnegieMellonU · 2026-11-14