CritPt eval reportedly so broken that Ant built a fixed version, per F5.1 system card
xeophon · x · 2026-09-11
Responding to a discussion about eval problems with CritPt and other hard benchmarks, xeophon claims CritPt is "so broken" that Ant has a fixed version of its own, which they use instead — as documented in the F5.1 system card.
More from Models
- GPT-6 Pro produces candidate proof for Erdős problem #488, passing two arithmetic checkers — basedjensen · 2026-09-11
- HighLevel claims early alpha access to rumored OpenAI GPT-Live-1, tests it in voice AI across 4M call insights — OpenAIDevs · 2026-09-11
- Polymarket opens betting on whether OpenAI's GPT-6 Astra loses public access — Polymarket · 2026-09-11
- Early users find OpenAI's GPT-6 Astra surprisingly good at generating SVG icons — floguo · 2026-09-11
- Google AI Search Leaks Its 'Hidden Thoughts' on Pet-Toxicity Conflict — real_maximpulse · 2026-09-11
- GPT-6 Astra's 99.9% ARC-AGI-3 Score Rejected by ARCPrize: Only 62.7% Under Standard Conditions — 新智元 · 2026-09-11