Decision Index benchmarks 275 models; some now natively supported in transformers
tomaarsen · x · 2026-11-10
Hugging Face engineer tomaarsen says he is excited about models producing consistent output structures for ML tasks. He points to the Decision Index, an extensive benchmark across 275 models, noting some are now natively supported in the transformers library.
Related event: Hugging Face Transformers Introduces Decision Models Primitive(6 posts)→
More from Research
- MKA cost breakdown: Claude Fable 5.2 spends $68 and 3 hours on a task Gemini 4 Carbon does for $8 — ArtificialAnlys · 2026-11-11
- MKA rubric design: top seven models cluster at 95–97% on Regression but span 28–57% on Climb — ArtificialAnlys · 2026-11-11
- MKA by output type: Fable 5.2 leads four file types, DeepSeek V4.1 Flash tops Text Files — ArtificialAnlys · 2026-11-11
- Inside an MKA task: auditing building plans costs Claude Fable 5.2 224 turns and ~$68 — ArtificialAnlys · 2026-11-11
- Google DeepMind and Artificial Analysis launch MKA benchmark; Claude Fable 5.2 tops at 57% Climb WSR — ArtificialAnlys · 2026-11-11
- MBZUAI releases open world model PAN 2: LLM planning plus 29B diffusion hits 75.1% step accuracy — rohanpaul_ai · 2026-11-11