Astra scores 98%+ on 10 of 37 benchmarks, nearly saturating eval suite
dswg97 · x · 2026-09-09
The evaluator flags a key caveat: Astra is close to saturating their benchmark suite, scoring ≥98% on 10 of 37 benchmarks, while GPT-5.5 saturates only 4/37. This makes confident time-horizon estimates difficult, and suggests existing evaluations are losing discriminative power for frontier models.
Related event: Astra nears benchmark saturation, scoring 98%+ on 10 of 37(2 posts)→
More from Models
- giffmana: the env being used in training is part of the point — giffmana · 2026-09-11
- awesome-llm-leaderboards: an open-source directory of LLM leaderboards, pricing tables, comparison tools — Last_Establishment_1 · 2026-09-11
- Anthropic claims it works to keep eval environments unidentifiable to models — MaxKannen · 2026-09-11
- Nex N2.5 Pro released on Hugging Face with 407GB of weights — jinnyjuice · 2026-09-11
- RoMa v2 image matching model unveiled in the usual black poster — ducha_aiki · 2026-09-11
- OpenAI rated Astra 'Critical' for cyber capabilities — and admits it's harder to monitor — theguywhobuilds · 2026-09-11