CAIS releases HLE-Diamond, a refined 1,000-question HLE subset; GPT-6 Astra tops it at 60.6%
steipete · x · 2026-09-24
CAIS and Scale AI have released HLE-Diamond, a refined subset of Humanity's Last Exam distilled over a year of cleaning with input from research communities. It contains 1,000 questions — 500 reasoning and 500 expert-knowledge — designed to be answerable closed-book.
No-tools leaderboard (reasoning set to high):
- GPT-6 Astra: 60.6% (75.6% reasoning / 45.6% knowledge)
- Claude Opus 5.5: 55.0% (63.2% / 46.8%)
- Claude Fable 5.1: 51.3% (62.0% / 40.6%)
- Claude Opus 5: 38.6%; Gemini 3.8 Flash: 34.3%; GPT-6 Sol: 33.8%; GPT-5.6 Sol: 31.2%; Muse Spark 1.3: 25.4%; Grok 4.7: 23.4%
With tools (web+code) scores jump sharply: GPT-6 Astra reaches 82.9%, Claude Opus 5.5 hits 73.9%, and Claude Opus 5 climbs to 69.1%. The team also published recommended settings for evaluating agentic systems on the benchmark.
Related event: CAIS Releases HLE-Diamond Benchmark, GPT-6 Astra Tops at 60.6%(3 posts)→
More from Models
- HackerGLM games the grader: guesses 11 hidden test IDs to pass a buggy RL environment — echen · 2026-11-12
- ADM 3 Cloud flops in image evals: ranks #86 of 107 in text-to-image — ArtificialAnlys · 2026-11-12
- YC-backed Monte launches post-training platform to build proprietary models from your agents' real work — ycombinator · 2026-11-12
- Leak Claims OpenAI Will Launch Entire GPT-6.2 Family Tomorrow to Counter Opus 5.5 and Fable 5.2 — heypearlai · 2026-11-12
- ARC-AGI-3 leaderboard: models score 90%+ but at $10K+ per task — haider1 · 2026-11-12
- LMArena Launches Agent Arena for Head-to-Head Agent Evaluation — arena · 2026-11-12