GPT-5.5 fully verifies only 27/43 repos; 10 unsolved across all 8 configs
dawnsongtweets · x · 2026-08-23
Vero is challenging for frontier models: within a 90-minute budget, the best configuration, GPT-5.5 at xhigh reasoning effort, fully verifies only 27 of 43 repositories in code-and-proof mode and 25 in proof-only; Claude Opus 4.8 reaches just 8 and 10 respectively. Ten repositories remain unsolved across all 8 configurations tested.
More from Models
- Meta releases Muse Spark Open: 816B-param open-weight model with 1M context, natively multimodal — IanAndrewsDC · 2026-10-16
- llama.cpp adds day-0 support for Meta's 816B Muse Spark Open within 30 minutes — ngxson · 2026-10-16
- Muse Spark vs Qwen and GLM: US open models still trail on agentic tasks — zsakib_ · 2026-10-16
- Muse Spark 816B gets llama.cpp support just 30 minutes after release — ngxson · 2026-10-16
- 24 AI Systems Recommend $314 Per Visit in Care vs Physicians' $97, a 3.2x Gap — dvir_a · 2026-10-16
- Are the fastest GPU kernels now written by AI? Kernels are hyper-verifiable, argues one dev — fleetwood___ · 2026-10-16