GPT-5.5 fully verifies only 27/43 repos; 10 unsolved across all 8 configs
dawnsongtweets · x · 2026-08-23
Vero is challenging for frontier models: within a 90-minute budget, the best configuration, GPT-5.5 at xhigh reasoning effort, fully verifies only 27 of 43 repositories in code-and-proof mode and 25 in proof-only; Claude Opus 4.8 reaches just 8 and 10 respectively. Ten repositories remain unsolved across all 8 configurations tested.
More from Models
- MAI-2 may have the largest publicly admitted training set size — rickasaurus · 2026-10-16
- Gemini 4 Argon built 5 apps from 5 prompts in 328s for $0.37 — aakashgupta · 2026-10-16
- Meta's coding-optimized open-weight Muse Spark Open lands on Modal with 1M context — AAAzzam · 2026-10-16
- Prism ML ships ternary Bonsai 2 models (2B/4B/9B) based on Qwen 3.5 — tcarambat · 2026-10-16
- Google launches Gemini 4 Argon frontier model on Cloud with 10 official demos — Saboo_Shubham_ · 2026-10-16
- RL practitioner: ~10,000 tasks still stump every frontier coding model — max_paperclips · 2026-10-16