AA-Omniscience v1.1: public set matches private benchmark at 0.99 correlation
ArtificialAnlys · x · 2026-10-31
Artificial Analysis updated its hallucination/knowledge benchmark to AA-Omniscience v1.1 and released a new public set on Hugging Face calibrated to match the difficulty of the private set.
At release, public and private Net Scores show a 0.99 correlation with a median difference of 0.0 points; accuracy and hallucination rate also correlate at 0.99, with median gaps of -0.07 and -0.29 percentage points. External researchers can now closely approximate the official leaderboard results using the public set.
Related event: Artificial Analysis Releases AA-Omniscience v1.1 Benchmark(2 posts)→
More from Research
- Real human brain cells are being used for AI, and Reddit is buzzing — EmotionalGuess9229 · 2026-10-31
- LLM judges harbor systematic errors that silently corrupt model rankings and training pipelines — burkov · 2026-10-31
- New paper quantifies ineffable concepts: 65 of 1,270 can never be taught to LLMs via language — sanmikoyejo · 2026-10-31
- Unsupervised Method Measures Whether LLM Messages Are Received — sanmikoyejo · 2026-10-31
- Neural net flies a drone in Acro mode from streamed game footage on a tiny board — yacineMTB · 2026-10-31
- llmlip-rs: Rust MLIP inference engine runs 6000-atom sims 17.7x faster than PyTorch MACE — CatAstro_Piyush · 2026-10-31