GPT-6 Astra tops ZeroBench, surpassing human baseline across all three metrics
Waiting4AniHaremFDVR · reddit · 2026-09-23
GPT-6 Astra makes a massive leap on ZeroBench, an extremely difficult vision benchmark, surpassing the human baseline on all three metrics. The poster clarifies the scoring: pass@5 counts if at least one of 5 attempts is correct; pass^5 requires all 5 correct (a reliability metric); and the reported "pass@1" is actually the average score across 5 attempts, not a true single-attempt pass@1.
Related event: GPT-6 Astra Reportedly Tops ZeroBench, Beating Human Baselines(2 posts)→
More from Models
- DeepMind launches AlphaGenome 2 with major performance leap in variant-effect prediction — anshulkundaje · 2026-11-11
- US data vendors accused of selling RL environments to Chinese military-linked firms — xeophon · 2026-11-11
- Theory: Typos Disrupt LLM Tokenization, Which May Explain Why AI Text Has a Distinct Smell — menhguin · 2026-11-11
- LMArena co-founder explains AI evaluation methodology and policy in new podcast — MattPerault · 2026-11-11
- Runway open-sources Praxis-1, turning video pretraining into real robot control — runwayml · 2026-11-11
- DeepMind launches AlphaGenome 2, spotting nearly 2x more gene-regulating mutations — GoogleDeepMind · 2026-11-11