SolidWorks-Bench: 73 Real Machines Tested, No AI Model Completes a Single Build
DevvMandal · x · 2026-11-19
SolidWorks-Bench: A Benchmark for CAD Agents
Markov released SolidWorks-Bench, testing whether multimodal agents can do a mechanical designer's job end to end inside SolidWorks. Each task is a real machine given as an engineering drawing package (general-arrangement drawing, fully dimensioned part drawings, supplier STEP files); the agent must model every custom part, assemble a working machine with mates, and save native editable files.
Headline result: no model completes a single machine.
- Scale: 73 machines, 3,434 custom parts, 5,959 bought parts, 12,213 drawing sheets; time limits from 2.7 to 24 hours
- Mean reward ranking: Fable 5.2 leads at 0.573, followed by Astra Sonnet 5.5 (0.426), Opus 5.5 (0.347), Kimi K3.1 (0.342), DeepSeek V4.1 (0.196), and others
- Even the best model gets only 43% of custom parts exactly right; models get close on individual parts but fail at assembly
- Tasks include a CNC router, binder-jetting 3D printer, FRC tracked robot, and autonomous robot
Related event: SolidWorks-Bench: No AI Can Assemble a Full Machine(2 posts)→
More from coding & agent
- Stripe engineer bans herself from slides, uses AI to build interactive presentations in 30 minutes — hazelcough · 2026-11-19
- Microsoft reveals Autopilot architecture with OpenClaw running in an untrusted sandbox — steipete · 2026-11-19
- Forward-deployed engineers: tie AI projects to a business metric before writing code — alex_verem · 2026-11-19
- Modem admits its AI PM agent flopped, rebuilds platform to work with your own agents — jevon · 2026-11-19
- Running an agent fleet across 15 repos: the bottleneck turned out to be the human — yi_ding · 2026-11-19
- Local MCP server lets Claude search your PDFs, old emails, voice memos and photos offline — IndependentBook5854 · 2026-11-19