Beam's Ten-Benchmark Breakdown: Doubles Peers on CritPt, Near-Bottom on Terminal-Bench
ArtificialAnlys · x · 2026-10-16
Artificial Analysis published Beam's full per-benchmark breakdown across the ten Intelligence Index evaluations:
- Strengths: AutomationBench-AA at 48.5% and CritPt at 13.1% (physics research reasoning) — more than double K2 Horizon (4.6%) and Qwen3.8 27B (5.4%) on the latter
- Weaknesses: Terminal-Bench 4.0 at just 3.0% and AA-Briefcase (agentic knowledge work) at 976 Elo
The accompanying charts cover AA-Omniscience accuracy vs hallucination rate, GDPval-AA, Humanity's Last Exam, GDPR.pdf, SciCode, AA-LCR v1.1 and more, letting readers locate Beam against open and proprietary models on every eval.
More from Models
- Opus 5.5 still leads a month later, but open-weight decision models are closing in — airesearch12 · 2026-10-16
- Google restricts Gemini Pro access for Google Plus subscribers, matching free tier — EmergencyLess8861 · 2026-10-16
- Meta releases Muse Spark Open, trained with vLLM-powered RL inference from day one — colinmcnamara · 2026-10-16
- Meta open-sources Muse Spark: runs on two Mac Studios, license bans passing output off as human-written — mawcopolow · 2026-10-16
- Golden Gate Claude explainer resurfaces: interpretability premise still holds — HarperSCarroll · 2026-10-16
- Hugging Face livestreams release of a new open-weight model — huggingface · 2026-10-16