CritPt v2 gains mostly from revised challenges, not tool access
ArtificialAnlys · x · 2026-10-23
Artificial Analysis clarifies that CritPt v2 scores are generally higher and not directly comparable to v1: GPT-6 Astra (max) went from 32% on v1 to 83% on v2. Their testing shows most of the increase comes from the revised challenges, with code-execution tool access adding a smaller gain. With leading models above 80%, v2 leaves less headroom at the frontier than v1 did.
Related event: CritPt v2 Score Gains Driven by Prompt Revisions, Not Tooling(2 posts)→
More from Models
- Microsoft says its AI diagnoses 4x more accurately than doctors, in Science — tomwarren · 2026-10-23
- OpenAI's Decisions API hits GA: near real-time routing up to 10x faster than GPT-6 Luna — OpenAIDevs · 2026-10-23
- DeepLearning.AI weekly: GPT-6.1 hits 97.9% byte parity on COBOL migration — DeepLearningAI · 2026-10-23
- Marker's lite mode scores 75.1 vs 42.3 for Azure Read on OmniParseBench — VikParuchuri · 2026-10-23
- Datalab launches lite OCR at $0.50 per 1,000 pages, 20-30x cheaper than Azure, Google and AWS layout APIs — VikParuchuri · 2026-10-23
- LiveNerf's 29-day data finds no nerfing of Claude Opus 5.5 — TheOnlyVibemaster · 2026-10-23