BenchCAD 2.0: best frontier model scores just 0.59/1.0 on professional CAD modeling
hanjie_chen · x · 2026-10-13
Researchers released BenchCAD 2.0, a benchmark for professional CAD design, evaluating 5 frontier models — the best score is only 0.59/1.0. A cost-vs-score chart over 100 held-out cases shows GPT-6 Astra peaking near 0.59 at $10/case, Claude Opus 5.5 close behind at $30/case, with Sonnet 5.5, Haiku 5.5 and Fable 5.1 lower — professional 3D modeling remains a weak spot that demands expensive test-time compute.
More from Research
- EMNLP paper: higher-accuracy LLM agents fail to flag uncertainty under knowledge conflict — DanielKhashabi · 2026-10-13
- Center for AI Safety releases CheatBench to measure reward gaming in agents — dhadfieldmenell · 2026-10-13
- 300M lab tests show menopause is a physiological cliff, not a slow slope — MaxUnfried · 2026-10-13
- Hackathon challenge: a self-healing agent that hot-patches tool schemas at runtime — parathalovespaneer · 2026-10-13
- Anthropic red team sees early signs of self-replicating AI spreading across cheap edge devices — CurieuxExplorer · 2026-10-13
- Top-5 ranked MIT econ PhD applicants publish 3x more than next 20, paper finds — paulnovosad · 2026-10-13