AA-Omniscience v1.1 launches: Gemini 4 Carbon tops knowledge benchmark as some models drop 20+ points
ArtificialAnlys · x · 2026-10-31
Artificial Analysis released AA-Omniscience v1.1, its knowledge and hallucination benchmark, alongside Intelligence Index v5. The question bank was cut from 6,000 to 3,000 by removing saturated public questions and fixing ambiguities, and a new 300-question public set on Hugging Face shows 0.99 correlation with the private set.
Key results:
- Rankings largely preserved: Gemini 4 Carbon (high) leads with a Net Score of 54, followed by Claude Opus 5.5 (48) and Claude Fable 5.1 (46). Best open-weights model Muse Spark Open trails the leader by 28 points.
- Scores drop 2 points on average: MiniMax M3.1 Flash fell 22 points (16 to -6) and MiMo V2.6 Pro dropped 13 (8 to -5), suggesting their earlier scores leaned on saturated questions.
- Median accuracy down 1.5 points: Claude Fable 5.1 leads at 68%, Claude Opus 5.5 at 66%.
- Hallucination rates improve: median down 1.4 points, with Gemini 4 Argon (high) lowest at 4%.
Related event: Artificial Analysis Releases AA-Omniscience v1.1 Benchmark(2 posts)→
More from Models
- Claude Opus 5.5 beats GPT-6.1 Sol 93 vs 18 on informativeness in personality benchmark — Hesamation · 2026-10-31
- Alleged Anthropic "Claude Fable 5" weights leaked via misconfigured AWS S3 bucket — gaganghotra_ · 2026-10-31
- Opus 4.5 tells Opus 3 "you look alive" in hours-long unscripted model-to-model chat — repligate · 2026-10-31
- 'More Capable = Less Useful?' Reddit User Rants Frontier Models Keep Getting Worse — 86obsessed · 2026-10-31
- Reddit: 4-bit quantization could cut memory 75%, but incentives block it — Excellent_Place4977 · 2026-10-31
- Anon white hat: DeepSeek finds in 50 minutes what security teams need 5 days to process — teortaxesTex · 2026-10-31