MiniMax-M3.1-Flash tops knowledge accuracy but hallucinates on 93% of unknowns
ArtificialAnlys · x · 2026-10-19
Artificial Analysis benchmarked MiniMax-M3.1-Flash on AA-Omniscience:
- Accuracy: 56%, ahead of GLM-5.3-Flash (28%) and DeepSeek V4.1 Flash 1017 (43%), in line with Gemini 3.8 Flash (55%)
- Aggressive answering strategy: it attempts 98% of questions and hallucinates on 93% of those it doesn't know, vs 28% (GLM), 67% (DeepSeek) and 55% (Gemini)
- Composite: AA-Omniscience Index of 16, beating GLM-5.3-Flash (8) and DeepSeek (5) but trailing Gemini 3.8 Flash (30)
- Sharp reversal from MiniMax-M3, which attempted only 35% of questions with an 18% hallucination rate but just 17% accuracy, scoring 1
Agentic results are mixed: 73% on AutomationBench-AA leads both rivals, but it trails on GDPval-AA (1458 Elo vs 1642/1746) and Terminal-Bench v4.0 (26% vs 33%/34%).
More from Models
- Users want open-format export from Claude Docs as charts only save as images — BlackHC · 2026-10-20
- EMNLP study: LLM safety interventions drop up to 4.5x when distress is wrapped in delusion — MaartenSap · 2026-10-20
- Cybersecurity-focused VLM OrcaCyber-Nano-1.0 trends on Hugging Face — orcarouter · 2026-10-20
- 0.8B local model with 2,000 labels beats Jev's frontier API on banking data — kalyan_kpl · 2026-10-20
- New TTS benchmark tests 26 models across 30,564 requests: fastest at 46ms — mattturck · 2026-10-20
- Agents handle unreleased language Jai flawlessly, even compile-time execution — lucasmeijer · 2026-10-20