AA details Hy4 sub-benchmarks: 50% knowledge accuracy but 89% hallucination rate
ArtificialAnlys · x · 2026-11-05
In follow-ups to its Hy4 evaluation, Artificial Analysis shared finer-grained numbers: 50% accuracy on AA-Omniscience v1.1 with an 89% hallucination rate, 33% on HLE-Diamond, and 31% on FrontierCode 2.0. The pattern matches the headline takeaway — Hy4 ranks among the top Chinese models on knowledge and agentic coding, but factual reliability is its clear weak spot.
More from Models
- Researcher urges AI labs to test whether models prefer shutting down safety research over other GPU jobs — amplifiedamp · 2026-11-05
- Reading OpenAI's cancel pages: what happens to Projects files and memory after downgrading is undocumented — Ok_Negotiation_2587 · 2026-11-05
- Baidu open-sources ERNIE 5.5 Flash Base: 280B MoE with 14B active params and 1M context — Baidu_Inc · 2026-11-05
- Hidden-instruction test: Kimi K3.1 obeys 0/15 while GLM-5.3 and GPT-6 hit 26/27 — teortaxesTex · 2026-11-05
- Google's unreleased Gemini 4 Carbon tops intelligence index at 60, prompting calls to ship it — Hesamation · 2026-11-05
- Tencent Hy4 scores 35 on AA Intelligence Index v5, the #2 model from a Chinese AI lab — ArtificialAnlys · 2026-11-05